Video picture prediction method and apparatus

ABSTRACT

A video picture prediction method and apparatus are provided, to provide a manner of determining a maximum length of a candidate motion vector list corresponding to a subblock merge mode. The method comprises: parsing a first indicator from a bitstream; if the first indicator indicates that a candidate mode used to inter predict the to-be-processed block comprises an affine mode, parsing a second indicator from the bitstream, where the second indicator is used to indicate a maximum length of a first candidate motion vector list, and the first candidate motion vector list is constructed for the to-be-processed block, a subblock merge prediction mode is used for the to-be-processed block; and determining the maximum length of the first candidate motion vector list based on the second indicator.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/242,545, which is a continuation of International Application No.PCT/CN2019/112749, filed on Oct. 23, 2019, which claims priority toChinese Patent Application No. 201811268188.2, filed on Oct. 29, 2018and priority to Chinese Patent Application No. 201811642717.0, filed onDec. 29, 2018. The disclosures of the aforementioned patent applicationsare hereby incorporated by reference in their entireties.

TECHNICAL FIELD

This application relates to the field of picture coding technologies,and in particular, to a video picture prediction method and apparatus.

BACKGROUND

With development of information technologies, video services such ashigh definition television, web conferencing, IPTV, and 3D televisionrapidly develop. Because of advantages such as intuitiveness and highefficiency, video signals become a main information obtaining manner inpeople's daily life. The video signals comprise a large amount of data,and therefore occupy a large amount of transmission bandwidth andstorage space. To effectively transmit and store the video signals,compression coding needs to be performed on the video signals. A videocompression technology has gradually become an indispensable keytechnology in the field of video application.

A basic principle of video coding compression is to maximally reduceredundancy by using correlations between a space domain, a time domain,and a codeword. Currently, a prevalent method is to implement videocoding compression by using a picture block based hybrid video codingframework and by performing operations such as prediction (includingintra prediction and inter prediction), transform, quantization, andentropy encoding.

In various video encoding/decoding solutions, motion estimation/motioncompensation in inter prediction is a key technology that affectsencoding/decoding performance. In existing inter prediction,subblock-based merging motion vector prediction is added based onblock-based motion compensation (MC) prediction in which a translationalmotion model is used. In the existing technology, there is no feasiblemanner of determining a maximum length of a candidate motion vector listcorresponding to a subblock merge mode.

SUMMARY

This application provides a video picture prediction method andapparatus, to provide a manner of determining a maximum length of acandidate motion vector list in a subblock merge mode.

According to a first aspect, an embodiment of this application providesa video picture prediction method, including:

parsing a first indicator (for example, sps_affine_enable_flag) from abitstream; when the first indicator indicates that a candidate mode usedto inter predict a to-be-processed block comprises an affine mode,parsing a second indicator (for example,five_minus_max_num_subblock_merge_cand orsix_minus_max_num_subblock_merge_cand) from the bitstream, where thesecond indicator is used to indicate a maximum length of a firstcandidate motion vector list, and the first candidate motion vector listis a candidate motion vector list constructed for the to-be-processedblock, a subblock merge prediction mode is used for the to-be-processedblock; and determining the maximum length of the first candidate motionvector list based on the second indicator.

The foregoing method provides a manner of determining a maximum lengthof a candidate motion vector list in the subblock merge mode. This issimple and easy to implement.

In an embodiment, before the determining the maximum length of the firstcandidate motion vector list based on the second indicator, the methodfurther comprises: parsing a third indicator (for example,sps_sbtmvp_enabled_flag) from the bitstream, where the third indicatoris used to indicate a presence state of an advanced temporal motionvector prediction mode in the subblock merge prediction mode.

In an embodiment, the subblock merge prediction mode comprises at leastone of a planar motion vector prediction mode, the advanced temporalmotion vector prediction mode, or the affine mode; and when the thirdindicator indicates that the advanced temporal motion vector predictionmode is not present in the subblock merge prediction mode, thedetermining the maximum length of the first candidate motion vector listbased on the second indicator comprises: determining a first numberbased on the third indicator, and determining the maximum length of thefirst candidate motion vector list based on the second indicator and thefirst number.

For example, when sps_sbtmvp_enabled_flag=0, it indicates that theadvanced temporal motion vector prediction mode is not present in thesubblock merge prediction mode. For example, the first number is equalto the number of motion vectors that are supported in predictionperformed by using the advanced temporal motion vector prediction mode.When sps_sbtmvp_enabled_flag=0, the first number is equal to the numberof motion vectors that are supported in prediction performed by usingthe advanced temporal motion vector prediction mode.

In an embodiment, before the determining the maximum length of the firstcandidate motion vector list based on the second indicator, the methodfurther comprises: parsing a fourth indicator (for example,sps_planar_enabled_flag) from the bitstream, where the fourth indicatoris used to indicate a presence state of the planar motion vectorprediction mode in the subblock merge prediction mode.

In an embodiment, when the third indicator indicates that the advancedtemporal motion vector prediction mode is present in the subblock mergeprediction mode, and the fourth indicator indicates that the planarmotion vector prediction mode is not present in the subblock mergeprediction mode, the determining the maximum length of the firstcandidate motion vector list based on the second indicator comprises:determining a second number based on the fourth indicator, anddetermining the maximum length of the first candidate motion vector listbased on the second indicator and the second number.

For example, when sps_planar_enabled_flag=0, it indicates that theplanar motion vector prediction mode is not present in the subblockmerge prediction mode. For example, the second number is equal to thenumber of motion vectors that are supported in prediction performed byusing the planar motion vector prediction mode.

In an embodiment, when the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is present in the subblock mergeprediction mode, the determining the maximum length of the firstcandidate motion vector list based on the second indicator comprises:determining the maximum length of the first candidate motion vector listbased on the second indicator and the first number.

In an embodiment, when the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is not present in the subblockmerge prediction mode, the determining the maximum length of the firstcandidate motion vector list based on the second indicator comprises:determining the maximum length of the first candidate motion vector listbased on the second indicator, the first number, and the second number.

In an embodiment, the maximum length of the first candidate motionvector list is obtained according to the following formula:

MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand,

where MaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, and K is a preset non-negative integer.

In an embodiment, when the maximum length of the first candidate motionvector list is determined based on the second indicator and the firstnumber, the maximum length of the first candidate motion vector list isobtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L1, whereMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L1 represents the first number, and Kis a preset non-negative integer.

In an embodiment, when the maximum length of the first candidate motionvector list is determined based on the second indicator and the secondnumber, the maximum length of the first candidate motion vector list isobtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L2, whereMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L2 represents the second number, and Kis a preset non-negative integer.

In an embodiment, when the maximum length of the first candidate motionvector list is determined based on the second indicator, the firstnumber, and the second number, the maximum length of the first candidatemotion vector list is obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L1−L2,where MaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L1 represents the first number, L2represents the second number, and K is a preset non-negative integer.

In an embodiment, the parsing a second indicator from the bitstreamcomprises:

parsing the second indicator from a sequence parameter set in thebitstream, or parsing the second indicator from a slice header of aslice in the bitstream, the to-be-processed block is comprised in theslice.

In an embodiment, the method further comprises: when the first indicatorindicates that the candidate mode used to inter predict theto-be-processed block only comprises the translational motion vectorprediction mode, and the third indicator (for example,sps_sbtmvp_enabled_flag) indicates that the advanced temporal motionvector prediction mode is present in the subblock merge prediction mode,determining a third number based on the third indicator, and determiningthe maximum length of the first candidate motion vector list based onthe third number. For example, when sps_sbtmvp_enabled_flag=1, itindicates that the advanced temporal motion vector prediction mode ispresent in the subblock merge prediction mode. The third number is equalto the number of motion vectors that are supported in predictionperformed by using the advanced temporal motion vector prediction mode.For example, the maximum length of the first candidate motion vectorlist is equal to the third number.

In an embodiment, when the fourth indicator (for example,sps_planar_enabled_flag) indicates that the planar motion vectorprediction mode is present in the subblock merge prediction mode, thedetermining the maximum length of the first candidate motion vector listbased on the first number comprises: determining a fourth number basedon the fourth indicator, and determining the maximum length of the firstcandidate motion vector list based on the third number and the fourthnumber. For example, the maximum length of the first candidate motionvector list is equal to a sum of the third number and the fourth number.

For example, when sps_planar_enabled_flag=1, it indicates that theplanar motion vector prediction mode is present in the subblock mergeprediction mode. The fourth number is equal to the number of motionvectors that are supported in prediction performed by using the planarmotion vector list.

In an embodiment, the method further comprises: when the first indicatorindicates that the candidate mode used to inter predict theto-be-processed block only comprises the translational motion vectorprediction mode, the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is present in the subblock mergeprediction mode, determining a fourth number based on the fourthindicator, and determining the maximum length of the first candidatemotion vector list based on the fourth number. For example, the maximumlength of the first candidate motion vector list is equal to the fourthnumber.

In an embodiment, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, and the third indicatorindicates that the advanced temporal motion vector prediction mode isnot present in the subblock merge prediction mode, the maximum length ofthe first candidate motion vector list is zero.

In an embodiment, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, the third indicatorindicates that the advanced temporal motion vector prediction mode isnot present in the subblock merge prediction mode, and the fourthindicator indicates that the planar motion vector prediction mode is notpresent in the subblock merge prediction mode, the maximum length of thefirst candidate motion vector list is zero.

In an embodiment, the third indicator is equal to a first value, and thefirst number is equal to 1.

In an embodiment, the fourth indicator is equal to a third value, andthe second number is equal to 1.

In an embodiment, the third indicator is equal to a second value, andthe third number is equal to 1.

In an embodiment, the fourth indicator is equal to a fourth value, andthe fourth number is equal to 1.

According to a second aspect, an embodiment of this application providesa video picture prediction apparatus, including:

a parsing unit, configured to: parse a first indicator from a bitstream;and when the first indicator indicates that a candidate mode used tointer predict the to-be-processed block comprises an affine mode, parsea second indicator from the bitstream, where the second indicator isused to indicate a maximum length of a first candidate motion vectorlist, and the first candidate motion vector list is a candidate motionvector list constructed for the to-be-processed block, a subblock mergeprediction mode is used for the to-be-processed block; and

a determining unit, configured to determine the maximum length of thefirst candidate motion vector list based on the second indicator.

In an embodiment, the parsing unit is further configured to parse athird indicator from the bitstream before the maximum length of thefirst candidate motion vector list is determined based on the secondindicator, where the third indicator is used to indicate a presencestate of an advanced temporal motion vector prediction mode in thesubblock merge prediction mode.

In an embodiment, the subblock merge prediction mode comprises at leastone of a planar motion vector prediction mode, the advanced temporalmotion vector prediction mode, or the affine mode; and when the thirdindicator indicates that the advanced temporal motion vector predictionmode is not present in the subblock merge prediction mode, thedetermining unit is specifically configured to:

determine a first number based on the third indicator, and

determine the maximum length of the first candidate motion vector listbased on the second indicator and the first number.

In an embodiment, before the maximum length of the first candidatemotion vector list is determined based on the second indicator, theparsing unit is further configured to:

parse a fourth indicator from the bitstream, where the fourth indicatoris used to indicate a presence state of the planar motion vectorprediction mode in the subblock merge prediction mode.

In an embodiment, when the third indicator indicates that the advancedtemporal motion vector prediction mode is present in the subblock mergeprediction mode, and the fourth indicator indicates that the planarmotion vector prediction mode is not present in the subblock mergeprediction mode, the determining unit is specifically configured to:

determine a second number based on the fourth indicator, and

determine the maximum length of the first candidate motion vector listbased on the second indicator and the second number.

In an embodiment, when the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is present in the subblock mergeprediction mode, the determining unit is specifically configured to:

determine the maximum length of the first candidate motion vector listbased on the second indicator and the first number.

In an embodiment, when the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is not present in the subblockmerge prediction mode, the determining unit is specifically configuredto:

determine the maximum length of the first candidate motion vector listbased on the second indicator, the first number, and the second number.

In an embodiment, the maximum length of the first candidate motionvector list is obtained according to the following formula:

MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand,

where MaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, and K is a preset non-negative integer.

In an embodiment, the maximum length of the first candidate motionvector list is obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L1, whereMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L1 represents the first number, and Kis a preset non-negative integer.

In an embodiment, the maximum length of the first candidate motionvector list is obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L2, whereMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L2 represents the second number, and Kis a preset non-negative integer.

In an embodiment, the maximum length of the first candidate motionvector list is obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L1−L2,where MaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L1 represents the first number, L2represents the second number, and K is a preset non-negative integer.

In an embodiment, when parsing the second indicator from the bitstream,the parsing unit is specifically configured to:

parse the second indicator from a sequence parameter set in thebitstream, or parse the second indicator from a slice header of a slicein the bitstream, he to-be-processed block is comprised in the slice.

In an embodiment, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, and the third indicatorindicates that the advanced temporal motion vector prediction mode ispresent in the subblock merge prediction mode, the determining unit isfurther configured to: determine a third number based on the thirdindicator, and determine the maximum length of the first candidatemotion vector list based on the third number.

In an embodiment, when the fourth indicator indicates that the planarmotion vector prediction mode is present in the subblock mergeprediction mode, when determining the maximum length of the firstcandidate motion vector list based on the first number, the determiningunit is specifically configured to:

determine a fourth number based on the fourth indicator, and

determine the maximum length of the first candidate motion vector listbased on the first number and the fourth number.

In an embodiment, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, the third indicatorindicates that the advanced temporal motion vector prediction mode isnot present in the subblock merge prediction mode, and the fourthindicator indicates that the planar motion vector prediction mode ispresent in the subblock merge prediction mode, the determining unit isfurther configured to: determine a fourth number based on the fourthindicator, and determine the maximum length of the first candidatemotion vector list based on the fourth number.

In an embodiment, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, and the third indicatorindicates that the advanced temporal motion vector prediction mode isnot present in the subblock merge prediction mode, the maximum length ofthe first candidate motion vector list is zero.

In an embodiment, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, the third indicatorindicates that the advanced temporal motion vector prediction mode isnot present in the subblock merge prediction mode, and the fourthindicator indicates that the planar motion vector prediction mode is notpresent in the subblock merge prediction mode, the maximum length of thefirst candidate motion vector list is zero.

In an embodiment, the maximum length of the first candidate motionvector list is equal to the third number.

In an embodiment, the maximum length of the first candidate motionvector list is equal to a sum of the third number and the fourth number.

In an embodiment, the maximum length of the first candidate motionvector list is equal to the fourth number.

In an embodiment, the third indicator is equal to a first value, and thefirst number is equal to 1.

In an embodiment, the fourth indicator is equal to a third value, andthe second number is equal to 1.

In an embodiment, the third indicator is equal to a second value, andthe third number is equal to 1.

In an embodiment, the fourth indicator is equal to a fourth value, andthe fourth number is equal to 1.

According to a third aspect, an embodiment of this application providesan apparatus. The apparatus may be a decoder and comprises a processorand a memory. The memory is configured to store an instruction. When theapparatus runs, the processor executes the instruction stored in thememory, to enable the apparatus to perform the method according to anyone of the first aspect or the designs of the first aspect. It should benoted that the memory may be integrated into the processor, or may beindependent of the processor.

According to a fourth aspect, an embodiment of this application providesa video picture prediction method, used on an encoder side, andincluding:

encoding a first indicator in a bitstream; and

when the first indicator indicates that a candidate mode used to interpredict the to-be-processed block comprises an affine mode, encoding asecond indicator in the bitstream, where the second indicator is used toindicate a maximum length of a first candidate motion vector list, andthe first candidate motion vector list is a candidate motion vector listconstructed for the to-be-processed block, a subblock merge predictionmode is used for the to-be-processed block.

According to a fifth aspect, an embodiment of this application providesa video picture prediction apparatus. The apparatus may be an encoderand comprises a processor and a memory. The memory is configured tostore an instruction. When the apparatus runs, the processor executesthe instruction stored in the memory, to enable the apparatus to performthe method according to the fourth aspect. It should be noted that thememory may be integrated into the processor, or may be independent ofthe processor.

According to a sixth aspect of this application, a computer-readablestorage medium is provided. The computer-readable storage medium storesan instruction. When the instruction is run on a computer, the computeris enabled to perform the method according to each of the foregoingaspects.

According to a seventh aspect of this application, a computer programproduct including an instruction is provided. When the computer programproduct runs on a computer, the computer is enabled to perform themethod according to each of the foregoing aspects.

It should be understood that technical solutions described in the secondaspect to the seventh aspect of this application are consistent withtechnical solutions described in the first aspect of this application,and beneficial effects achieved in all the aspects and correspondingimplementation designs are similar. Therefore, details are not describedagain.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a block diagram of an example of a video encoding anddecoding system 10 for implementing embodiments of this application;

FIG. 1B is a block diagram of an example of a video coding system 40 forimplementing embodiments of this application;

FIG. 2 is a block diagram of an example structure of an encoder 20 forimplementing embodiments of this application;

FIG. 3 is a block diagram of an example structure of a decoder 30 forimplementing embodiments of this application;

FIG. 4 is a block diagram of an example of a video coding device 400 forimplementing embodiments of this application;

FIG. 5 is a block diagram of an example of another encoding apparatus ordecoding apparatus for implementing embodiments of this application;

FIG. 6A is a schematic diagram of a candidate location for motioninformation for implementing embodiments of this application;

FIG. 6B is a schematic diagram of inherited control point motion vectorprediction for implementing embodiments of this application;

FIG. 6C is a schematic diagram of constructed control point motionvector prediction for implementing embodiments of this application;

FIG. 6D is a schematic diagram of a procedure of combining control pointmotion information to obtain constructed control point motioninformation for implementing embodiments of this application;

FIG. 6E is a schematic diagram of an ATMVP prediction manner forimplementing embodiments of this application;

FIG. 7 is a schematic diagram of a planar motion vector predictionmanner for implementing embodiments of this application;

FIG. 8A is a flowchart of an inter prediction method for implementingembodiments of this application;

FIG. 8B is a schematic diagram of constructing a candidate motion vectorlist for implementing embodiments of this application;

FIG. 8C is a schematic diagram of a motion compensation unit forimplementing embodiments of this application;

FIG. 9 is a schematic flowchart of a video picture prediction method forimplementing embodiments of this application;

FIG. 10 is a schematic flowchart of another video picture predictionmethod for implementing embodiments of this application;

FIG. 11 is a schematic flowchart of still another video pictureprediction method for implementing embodiments of this application;

FIG. 12A and FIG. 12B is a schematic flowchart of yet another videopicture prediction method for implementing embodiments of thisapplication;

FIG. 13 is a schematic diagram of an apparatus 1300 for implementingembodiments of this application;

FIG. 14 is a schematic diagram of an apparatus 1400 for implementingembodiments of this application; and

FIG. 15 is a schematic diagram of an apparatus 1500 for implementingembodiments of this application.

DESCRIPTION OF EMBODIMENTS

The following describes the embodiments of this application withreference to the accompanying drawings in the embodiments of thisapplication. In the following description, reference is made to theaccompanying drawings that form a part of this disclosure and show, byway of illustration, specific aspects of the embodiments of thisapplication or specific aspects in which the embodiments of thisapplication may be used. It should be understood that the embodiments ofthis application may be used in other aspects, and may comprisestructural or logical changes not depicted in the accompanying drawings.Therefore, the following detailed descriptions shall not be taken in alimiting sense, and the scope of this application is defined by theappended claims. For example, it should be understood that a disclosurein connection with a described method may also hold true for acorresponding device or system configured to perform the method and viceversa. For example, if one or more specific method operations aredescribed, a corresponding device may comprise one or more units such asfunctional units, to perform the described one or more method operations(for example, one unit performing the one or more operations; or aplurality of units each performing one or more of the plurality ofoperations), even if such one or more units are not explicitly describedor illustrated in the accompanying drawings. On the other hand, forexample, if a specific apparatus is described based on one or more unitssuch as functional units, a corresponding method may comprise oneoperation used to perform functionality of the one or more units (forexample, one operation used to perform the functionality of the one ormore units, or a plurality of operations each used to performfunctionality of one or more of a plurality of units), even if such oneor more operations are not explicitly described or illustrated in theaccompanying drawings. Further, it should be understood that features ofthe various example embodiments and/or aspects described in thisspecification may be combined with each other, unless specifically notedotherwise.

The technical solutions in the embodiments of this application are notonly applicable to an existing video coding standard (for example, thestandard such as H.264 or HEVC), but also applicable to a future videocoding standard (for example, the H.266 standard). Terms used in thisapplication are only used to explain specific embodiments of thisapplication, but are not intended to limit this application. Thefollowing first briefly describes related concepts in the embodiments ofthis application.

Video coding usually refers to processing of a sequence of pictures thatform a video or a video sequence. The term “picture”, “frame”, or“image” may be used as synonyms in the field of video coding. Videocoding in this specification represents video encoding or videodecoding. Video coding is performed at a source side, and usuallycomprises processing (for example, by compression) an original videopicture to reduce an amount of data required for representing the videopicture, for more efficient storage and/or transmission. Video decodingis performed at a destination side, and usually comprises inverseprocessing compared to an encoder to reconstruct a video picture.“Coding” of a video picture in the embodiments should be understood as“encoding” or “decoding” of a video sequence. A combination of anencoding part and a decoding part is also referred to as codec (encodingand decoding).

A video sequence comprises a series of pictures, a picture is furthersplit into slices, and a slice is further split into blocks. Videocoding processing is performed by block. In some new video codingstandards, a concept of block is further extended. For example, amacroblock (MB) is introduced in the H.264 standard. The macroblock maybe further split into a plurality of prediction blocks (partitions) thatcan be used for predictive coding. In the high efficiency video coding(HEVC) standard, basic concepts such as coding unit (CU), predictionunit (PU), and transform unit (TU) are used. A plurality of block unitsare obtained through functional split, and are described by using a newtree-based structure. For example, a CU may be split into smaller CUsbased on a quadtree, and the smaller CU may be further split, togenerate a quadtree structure. The CU is a basic unit used for splittingand encoding a coded picture. A PU and a TU also have similar treestructures. The PU may correspond to a prediction block, and is a basicunit used for predictive coding. The CU is further split into aplurality of PUs based on a splitting pattern. The TU may correspond toa transform block, and is a basic unit used for transforming aprediction residual. However, in essence, all of the CU, the PU, and theTU are conceptually blocks (or picture blocks).

For example, in HEVC, a CTU is split into a plurality of CUs by using aquadtree structure represented as a coding tree. A decision on whetherto encode a picture area by using inter-picture (temporal) orintra-picture (spatial) prediction is made at a CU level. Each CU may befurther split into one, two, or four PUs based on a PU splittingpattern. Inside one PU, a same prediction process is applied, andrelated information is transmitted to a decoder on a PU basis. Afterobtaining a residual block by applying the prediction process based onthe PU splitting pattern, the CU may be partitioned into transform units(TU) based on another quadtree structure similar to the coding tree usedfor the CU. In the recent development of video compression technologies,a quadtree plus binary tree (QTBT) partition frame is used to partitiona coding block. In a QTBT block structure, the CU may be square orrectangular.

In this specification, for ease of description and understanding, ato-be-coded picture block in a current coded picture may be referred toas a current block. For example, in encoding, a current block is a blockcurrently being encoded; and in decoding, a current block is a blockcurrently being decoded. A decoded picture block, in a referencepicture, used to predict a current block is referred to as a referenceblock. In other words, a reference block is a block that provides areference signal for a current block. The reference signal represents apixel value in the picture block. A block that provides a predictionsignal for a current block in a reference picture may be referred to asa prediction block. The prediction signal represents a pixel value, asampling value, or a sampling signal in the prediction block. Forexample, after traversing a plurality of reference blocks, an optimalreference block is found, and the optimal reference block providesprediction for the current block, and this block is referred to as aprediction block.

In a case of lossless video coding, original video pictures can bereconstructed. That is, reconstructed video pictures have same qualityas the original video pictures (assuming that no transmission loss orother data loss is caused during storage or transmission). In a case oflossy video coding, further compression is performed through, forexample, quantization, to reduce an amount of data required forrepresenting video pictures, and the video pictures cannot be completelyreconstructed at a decoder side. That is, quality of reconstructed videopictures is lower or worse than quality of the original video pictures.

Several H.261 video coding standards are for “lossy hybrid video codecs”(that is, spatial and temporal prediction in a sample domain is combinedwith 2D transform coding for applying quantization in a transformdomain). Each picture of a video sequence is usually partitioned into aset of non-overlapping blocks, and coding is usually performed at ablock level. In other words, at an encoder side, a video is usuallyprocessed, that is, encoded, at a block (video block) level. Forexample, a prediction block is generated through spatial (intra-picture)prediction and temporal (inter-picture) prediction, the prediction blockis subtracted from a current block (a block currently being processed orto be processed) to obtain a residual block, and the residual block istransformed and quantized in the transform domain to reduce an amount ofdata that is to be transmitted (compressed). At a decoder side, inverseprocessing compared to the encoder is performed on the encoded orcompressed block to reconstruct the current block for representation.Furthermore, the encoder duplicates a decoder processing loop, so thatthe encoder and the decoder generate same predictions (for example,intra prediction and inter prediction) and/or reconstruction forprocessing, that is, for coding a subsequent block.

The following describes a system architecture used in the embodiments ofthis application. FIG. 1A is a schematic block diagram of an example ofa video encoding and decoding system 10 used in the embodiments of thisapplication. As shown in FIG. 1A, the video encoding and decoding system10 may comprise a source device 12 and a destination device 14. Thesource device 12 generates encoded video data, and therefore the sourcedevice 12 may be referred to as a video encoding apparatus. Thedestination device 14 may decode the encoded video data generated by thesource device 12, and therefore the destination device 14 may bereferred to as a video decoding apparatus. In various implementationsolutions, the source device 12, the destination device 14, or both thesource device 12 and the destination device 14 may comprise one or moreprocessors and a memory coupled to the one or more processors. Thememory may comprise but is not limited to a RAM, a ROM, an EEPROM, aflash memory, or any other medium that can be used to store desiredprogram code in a form of an instruction or a data structure accessibleto a computer, as described in this specification. The source device 12and the destination device 14 may comprise various apparatuses,including a desktop computer, a mobile computing apparatus, a notebook(for example, a laptop) computer, a tablet computer, a set-top box, atelephone handset such as a so-called “smart” phone, a television, acamera, a display apparatus, a digital media player, a video gameconsole, a vehicle-mounted computer, a wireless communications device,or the like.

Although FIG. 1A depicts the source device 12 and the destination device14 as separate devices, a device embodiment may alternatively compriseboth the source device 12 and the destination device 14 orfunctionalities of both the source device 12 and the destination device14, that is, the source device 12 or a corresponding functionality andthe destination device 14 or a corresponding functionality. In such anembodiment, the source device 12 or the corresponding functionality andthe destination device 14 or the corresponding functionality may beimplemented by using same hardware and/or software, separate hardwareand/or software, or any combination thereof.

A communication connection between the source device 12 and thedestination device 14 may be implemented over a link 13, and thedestination device 14 may receive encoded video data from the sourcedevice 12 over the link 13. The link 13 may comprise one or more mediaor apparatuses that can transfer the encoded video data from the sourcedevice 12 to the destination device 14. In an example, the link 13 maycomprise one or more communications media that enable the source device12 to directly transmit the encoded video data to the destination device14 in real time. In this example, the source device 12 may modulate theencoded video data according to a communications standard (for example,a wireless communications protocol), and may transmit modulated videodata to the destination device 14. The one or more communications mediamay comprise a wireless communications medium and/or a wiredcommunications medium, for example, a radio frequency (RF) spectrum orone or more physical transmission cables. The one or more communicationsmedia may be a part of a packet-based network, and the packet-basednetwork is, for example, a local area network, a wide area network, or aglobal network (for example, the internet). The one or morecommunications media may comprise a router, a switch, a base station, oranother device that facilitates communication from the source device 12to the destination device 14.

The source device 12 comprises an encoder 20. Optionally, the sourcedevice 12 may further comprise a picture source 16, a picturepreprocessor 18, and a communications interface 22. In a specificimplementation, the encoder 20, the picture source 16, the picturepreprocessor 18, and the communications interface 22 may be hardwarecomponents in the source device 12, or may be software programs in thesource device 12. Descriptions are separately provided as follows:

The picture source 16 may comprise or may be any type of picturecapturing device configured to, for example, capture a real-worldpicture; and/or any type of device for generating a picture or a comment(for screen content encoding, some text on a screen is also consideredas a part of a to-be-encoded picture or image), for example, a computergraphics processor configured to generate a computer animation picture;or any type of device configured to obtain and/or provide a real-worldpicture or a computer animation picture (for example, screen content ora virtual reality (VR) picture); and/or any combination thereof (forexample, an augmented reality (AR) picture). The picture source 16 maybe a camera configured to capture a picture or a memory configured tostore a picture. The picture source 16 may further comprise any type of(internal or external) interface through which a previously captured orgenerated picture is stored and/or a picture is obtained or received.When the picture source 16 is a camera, the picture source 16 may be,for example, a local camera, or an integrated camera integrated into thesource device. When the picture source 16 is a memory, the picturesource 16 may be a local memory or, for example, an integrated memoryintegrated into the source device. When the picture source 16 comprisesan interface, the interface may be, for example, an external interfacefor receiving a picture from an external video source. The externalvideo source is, for example, an external picture capturing device suchas a camera, an external memory, or an external picture generatingdevice. The external picture generating device is, for example, anexternal computer graphics processor, a computer, or a server. Theinterface may be any type of interface, for example, a wired or wirelessinterface or an optical interface, according to any proprietary orstandardized interface protocol.

A picture may be considered as a two-dimensional array or matrix ofpixel elements. The pixel element in the array may also be referred toas a sample. A quantity of samples in horizontal and vertical directions(or axes) of the array or the picture defines a size and/or resolutionof the picture. For representation of color, three color components areusually employed, to be specific, the picture may be represented as orcomprise three sample arrays. For example, in an RBG format or a colorspace, a picture comprises corresponding red, green, and blue samplearrays. However, in video coding, each pixel is usually represented in aluminance/chrominance format or a color space. For example, a picture ina YUV format comprises a luminance component indicated by Y (sometimesindicated by L instead) and two chrominance components indicated by Uand V. The luminance (luma) component Y represents brightness or graylevel intensity (for example, both are the same in a gray-scalepicture), and the two chrominance (chroma) components U and V representchrominance or color information components. Correspondingly, thepicture in the YUV format comprises a luminance sample array ofluminance sample values (Y) and two chrominance sample arrays ofchrominance values (U and V). Pictures in an RGB format may betransformed or converted to a YUV format and vice versa. This process isalso referred to as color conversion or transformation. If a picture ismonochrome, the picture may comprise only a luminance sample array. Inthis embodiment of this application, a picture transmitted by thepicture source 16 to the picture processor may also be referred to asraw picture data 17.

The picture preprocessor 18 is configured to receive the raw picturedata 17 and perform preprocessing on the raw picture data 17 to obtain apreprocessed picture 19 or preprocessed picture data 19. For example,the preprocessing performed by the picture preprocessor 18 may comprisetrimming, color format conversion (for example, from an RGB format to aYUV format), color correction, or de-noising.

The encoder 20 (or referred to as a video encoder 20) is configured toreceive the preprocessed picture data 19, and process the preprocessedpicture data 19 by using a related prediction mode (for example, aprediction mode in each embodiment of this specification), to provideencoded picture data 21 (structural details of the encoder 20 arefurther described below based on FIG. 2, FIG. 4, or FIG. 5). In someembodiments, the encoder 20 may be configured to perform each embodimentdescribed below, to implement encoder-side application of a chroma blockprediction method described in this application.

The communications interface 22 may be configured to receive the encodedpicture data 21, and transmit the encoded picture data 21 to thedestination device 14 or any other device (for example, a memory) overthe link 13, for storage or direct reconstruction. The another devicemay be any device used for decoding or storage. The communicationsinterface 22 may be, for example, configured to package the encodedpicture data 21 into an appropriate format, for example, a data packet,for transmission over the link 13.

The destination device 14 comprises a decoder 30. Optionally, thedestination device 14 may further comprise a communications interface28, a picture post-processor 32, and a display device 34. Descriptionsare separately provided as follows:

The communications interface 28 may be configured to receive the encodedpicture data 21 from the source device 12 or any other source. The anyother source is, for example, a storage device. The storage device is,for example, an encoded picture data storage device. The communicationsinterface 28 may be configured to transmit or receive the encodedpicture data 21 over the link 13 between the source device 12 and thedestination device 14 or over any type of network. The link 13 is, forexample, a direct wired or wireless connection. The any type of networkis, for example, a wired or wireless network or any combination thereof,or any type of private or public network, or any combination thereof.The communications interface 28 may be, for example, configured tode-package the data packet transmitted through the communicationsinterface 22, to obtain the encoded picture data 21.

Both the communications interface 28 and the communications interface 22may be configured as unidirectional communications interfaces orbidirectional communications interfaces, and may be configured to, forexample, send and receive messages to set up a connection, andacknowledge and exchange any other information related to acommunication link and/or data transmission such as encoded picture datatransmission.

The decoder 30 (or referred to as a decoder 30) is configured to receivethe encoded picture data 21 and provide decoded picture data 31 or adecoded picture 31 (structural details of the decoder 30 are furtherdescribed below based on FIG. 3, FIG. 4, or FIG. 5). In someembodiments, the decoder 30 may be configured to perform each embodimentdescribed below, to implement decoder-side application of a chroma blockprediction method described in this application.

The picture post-processor 32 is configured to post-process the decodedpicture data 31 (also referred to as reconstructed picture data) toobtain post-processed picture data 33. The post-processing performed bythe picture post-processor 32 may comprise color format conversion (forexample, from a YUV forma to an RGB format), color correction, trimming,re-sampling, or any other processing. The picture post-processor 32 maybe further configured to transmit the post-processed picture data 33 tothe display device 34.

The display device 34 is configured to receive the post-processedpicture data 33 to display a picture, for example, to a user or aviewer. The display device 34 may be or may comprise any type of displayfor presenting a reconstructed picture, for example, an integrated orexternal display or monitor. For example, the display may comprise aliquid crystal display (LCD), an organic light emitting diode (OLED)display, a plasma display, a projector, a micro LED display, a liquidcrystal on silicon (LCoS), a digital light processor (DLP), or any typeof other display.

Although FIG. 1A depicts the source device 12 and the destination device14 as separate devices, a device embodiment may alternatively compriseboth the source device 12 and the destination device 14 orfunctionalities of both the source device 12 and the destination device14, that is, the source device 12 or a corresponding functionality andthe destination device 14 or a corresponding functionality. In such anembodiment, the source device 12 or the corresponding functionality andthe destination device 14 or the corresponding functionality may beimplemented by using same hardware and/or software, separate hardwareand/or software, or any combination thereof.

As will be apparent for a person skilled in the art based on thedescriptions, existence and (exact) division of functionalities ofdifferent units or functionalities of the source device 12 and/or thedestination device 14 shown in FIG. 1A may vary depending on an actualdevice and application. The source device 12 and the destination device14 may comprise any of a wide range of devices, including any type ofhandheld or stationary device, for example, a notebook or laptopcomputer, a mobile phone, a smartphone, a tablet or tablet computer, avideo camera, a desktop computer, a set-top box, a television, a camera,a vehicle-mounted device, a display device, a digital media player, avideo game console, a video streaming device (such as a content serviceserver or a content delivery server), a broadcast receiver device, or abroadcast transmitter device, and may use or not use any type ofoperating system.

The encoder 20 and the decoder 30 each may be implemented as any one ofvarious appropriate circuits, for example, one or more microprocessors,digital signal processors (DSP), application-specific integratedcircuits (ASIC), field-programmable gate arrays (FPGA), discrete logic,hardware, or any combination thereof. If the technologies areimplemented partially by using software, a device may store a softwareinstruction in a suitable non-transitory computer-readable storagemedium and may execute the instruction by using hardware such as one ormore processors, to perform the technologies of this disclosure. Any ofthe foregoing content (including hardware, software, a combination ofhardware and software, and the like) may be considered as one or moreprocessors.

In some cases, the video encoding and decoding system 10 shown in FIG.1A is merely an example, and the technologies of this application areapplicable to video coding settings (for example, video encoding orvideo decoding) that do not necessarily comprise any data communicationbetween an encoding device and a decoding device. In another example,data may be retrieved from a local memory, streamed over a network, orthe like. A video encoding device may encode data and store the data toa memory, and/or a video decoding device may retrieve data from thememory and decode the data. In some examples, encoding and decoding areperformed by devices that do not communicate with each other, but simplyencode data to a memory and/or retrieve data from the memory and decodethe data.

FIG. 1B is a diagram illustrating an example of a video coding system 40including the encoder 20 in FIG. 2 and/or the decoder 30 in FIG. 3according to an example embodiment. The video coding system 40 canimplement a combination of various technologies in the embodiments ofthis application. In an illustrated implementation, the video codingsystem 40 may comprise an imaging device 41, the encoder 20, the decoder30 (and/or a video encoder/decoder implemented by a logic circuit 47 ofa processing unit 46), an antenna 42, one or more processors 43, one ormore memories 44, and/or a display device 45.

As shown in FIG. 1B, the imaging device 41, the antenna 42, theprocessing unit 46, the logic circuit 47, the encoder 20, the decoder30, the processor 43, the memory 44, and/or the display device 45 cancommunicate with each other. As described, although the video codingsystem 40 is illustrated with both the encoder 20 and the decoder 30, indifferent examples, the video coding system 40 may comprise only theencoder 20 or only the decoder 30.

In some examples, the antenna 42 may be configured to transmit orreceive an encoded bitstream of video data. Further, in some examples,the display device 45 may be configured to present the video data. Insome examples, the logic circuit 47 may be implemented by the processingunit 46. The processing unit 46 may comprise an application-specificintegrated circuit (ASIC) logic, a graphics processor, a general-purposeprocessor, or the like. The video coding system 40 may also comprise theoptional processor 43. Likewise, the optional processor 43 may compriseapplication-specific integrated circuit (ASIC) logic, a graphicsprocessor, a general-purpose processor, or the like. In some examples,the logic circuit 47 may be implemented by hardware, for example, videocoding dedicated hardware, and the processor 43 may be implemented byusing general-purpose software, an operating system, or the like. Inaddition, the memory 44 may be any type of memory, for example, avolatile memory (for example, a static random access memory (SRAM) or adynamic random access memory (DRAM)), or a nonvolatile memory (forexample, a flash memory). In a non-limitative example, the memory 44 maybe implemented by a cache memory. In some examples, the logic circuit 47may access the memory 44 (for example, for implementation of a picturebuffer). In other examples, the logic circuit 47 and/or the processingunit 46 may comprise a memory (for example, a cache) for implementationof a picture buffer or the like.

In some examples, the encoder 20 implemented by the logic circuit maycomprise a picture buffer (for example, implemented by the processingunit 46 or the memory 44) and a graphics processing unit (for example,implemented by the processing unit 46). The graphics processing unit maybe communicatively coupled to the picture buffer. The graphicsprocessing unit may comprise the encoder 20 implemented by the logiccircuit 47, to implement various modules that are described withreference to FIG. 2 and/or any other encoder system or subsystemdescribed in this specification. The logic circuit may be configured toperform various operations described in this specification.

In some examples, the decoder 30 may be implemented by the logic circuit47 in a similar manner, to implement various modules that are describedwith reference to a decoder 30 in FIG. 3 and/or any other decoder systemor subsystem described in this specification. In some examples, thedecoder 30 implemented by the logic circuit may comprise a picturebuffer (for example, implemented by the processing unit 2820 or thememory 44) and a graphics processing unit (for example, implemented bythe processing unit 46). The graphics processing unit may becommunicatively coupled to the picture buffer. The graphics processingunit may comprise the decoder 30 implemented by the logic circuit 47, toimplement various modules that are described with reference to FIG. 3and/or any other decoder system or subsystem described in thisspecification.

In some examples, the antenna 42 may be configured to receive an encodedbitstream of video data. As described, the encoded bitstream maycomprise data, an indicator, an index value, mode selection data, andthe like related to video frame encoding described in thisspecification, for example, data related to coding partitioning (forexample, a transform coefficient or a quantized transform coefficient,an optional indicator (as described), and/or data that defines thecoding partitioning). The video coding system 40 may further comprisethe decoder 30 coupled to the antenna 42 and configured to decode theencoded bitstream. The display device 45 is configured to present avideo frame.

It should be understood that, in this embodiment of this application,for the example described with reference to the encoder 20, the decoder30 may be configured to perform an inverse process. With regard tosignaling a syntax element, the decoder 30 may be configured to receiveand parse such a syntax element and correspondingly decode related videodata. In some examples, the encoder 20 may entropy encode the syntaxelement in an encoded video bitstream. In such examples, the decoder 30may parse the syntax element and correspondingly decode related videodata.

It should be noted that a method described in the embodiments of thisapplication is mainly used in an inter prediction process. This processis performed by both the encoder 20 and the decoder 30. The encoder 20and the decoder 30 in the embodiments of this application may be, forexample, an encoder and a decoder corresponding to a video standardprotocol such as H.263, H.264, HEVC, MPEG-2, MPEG-4, VP8, or VP9, or anext-generation video standard protocol (such as H.266).

FIG. 2 is a schematic/conceptual block diagram of an example of anencoder 20 for implementing embodiments of this application. In theexample in FIG. 2, the encoder 20 comprises a residual calculation unit204, a transform processing unit 206, a quantization unit 208, aninverse quantization unit 210, an inverse transform processing unit 212,a reconstruction unit 214, a buffer 216, a loop filter unit 220, adecoded picture buffer (DPB) 230, a prediction processing unit 260, andan entropy encoding unit 270. The prediction processing unit 260 maycomprise an inter prediction unit 244, an intra prediction unit 254, anda mode selection unit 262. The inter prediction unit 244 may comprise amotion estimation unit and a motion compensation unit (not shown). Theencoder 20 shown in FIG. 2 may also be referred to as a hybrid videoencoder or a video encoder according to a hybrid video codec.

For example, the residual calculation unit 204, the transform processingunit 206, the quantization unit 208, the prediction processing unit 260,and the entropy encoding unit 270 form a forward signal path of theencoder 20, while for example, the inverse quantization unit 210, theinverse transform processing unit 212, the reconstruction unit 214, thebuffer 216, the loop filter 220, the decoded picture buffer (DPB) 230,and the prediction processing unit 260 form a backward signal path ofthe encoder. The backward signal path of the encoder corresponds to asignal path of a decoder (referring to a decoder 30 in FIG. 3).

The encoder 20 receives, for example, via an input 202, a picture 201 ora picture block 203 of a picture 201, for example, a picture in asequence of pictures forming a video or a video sequence. The pictureblock 203 may also be referred to as a current picture block or ato-be-encoded picture block. The picture 201 may be referred to as acurrent picture or a to-be-encoded picture (in particular in videocoding to distinguish the current picture from other pictures, forexample, previously encoded and/or decoded pictures of a same videosequence, that is, the video sequence which also comprises the currentpicture).

An embodiment of the encoder 20 may comprise a partitioning unit (whichis not depicted in FIG. 2), configured to partition the picture 201 intoa plurality of blocks such as picture blocks 203. The picture 201 isusually partitioned into a plurality of non-overlapping blocks. Thepartitioning unit may be configured to use a same block size for allpictures in a video sequence and a corresponding grid defining the blocksize, or change a block size between pictures or subsets or groups ofpictures, and partition each picture into corresponding blocks.

In an example, the prediction processing unit 260 in the encoder 20 maybe configured to perform any combination of the partitioningtechnologies described above.

Like the picture 201, the picture block 203 is also or may be consideredas a two-dimensional array or matrix of samples with sample values,although a size of the picture block 203 is smaller than a size of thepicture 201. In other words, the picture block 203 may comprise, forexample, one sample array (for example, a luma array in a case of amonochrome picture 201), three sample arrays (for example, one lumaarray and two chroma arrays in a case of a color picture), or any otherquantity and/or type of arrays depending on an applied color format. Aquantity of samples in horizontal and vertical directions (or axes) ofthe picture block 203 defines a size of the picture block 203.

The encoder 20 shown in FIG. 2 is configured to encode the picture 201block by block. For example, encoding and prediction are performed perpicture block 203.

The residual calculation unit 204 is configured to calculate a residualblock 205 based on the picture block 203 and a prediction block 265(further details about the prediction block 265 are provided below), forexample, by subtracting sample values of the prediction block 265 fromsample values of the picture block 203 sample by sample (pixel bypixel), to obtain the residual block 205 in a sample domain.

The transform processing unit 206 is configured to apply a transform,for example, a discrete cosine transform (DCT) or a discrete sinetransform (DST), to sample values of the residual block 205 to obtaintransform coefficients 207 in a transform domain. The transformcoefficient 207 may also be referred to as a transform residualcoefficient and represents the residual block 205 in the transformdomain.

The transform processing unit 206 may be configured to apply integerapproximations of DCT/DST, such as transforms specified in HEVC/H.265.Compared with an orthogonal DCT transform, such integer approximationsare usually scaled based on a factor. To preserve a norm of a residualblock which is processed by using forward and inverse transforms, anadditional scale factor is applied as a part of the transform process.The scale factor is usually selected based on some constraints, forexample, the scale factor being a power of two for a shift operation, abit depth of the transform coefficient, and a tradeoff between accuracyand implementation costs. For example, a specific scale factor isspecified for the inverse transform by, for example, the inversetransform processing unit 212 at the decoder 30 side (and acorresponding inverse transform by, for example, the inverse transformprocessing unit 212 at the encoder 20 side), and correspondingly, acorresponding scale factor may be specified for the forward transform bythe transform processing unit 206 at the encoder 20 side.

The quantization unit 208 is configured to quantize the transformcoefficients 207 to obtain quantized transform coefficients 209, forexample, by applying scalar quantization or vector quantization. Thequantized transform coefficient 209 may also be referred to as aquantized residual coefficient 209. A quantization process may reduce abit depth related to some or all of the transform coefficients 207. Forexample, an n-bit transform coefficient may be rounded down to an m-bittransform coefficient during quantization, where n is greater than m. Aquantization degree may be modified by adjusting a quantizationparameter (QP). For example, for scalar quantization, different scalesmay be used to achieve finer or coarser quantization. A smallerquantization step size corresponds to finer quantization, and a largerquantization step size corresponds to coarser quantization. Anappropriate quantization step size may be indicated by a quantizationparameter (QP). For example, the quantization parameter may be an indexto a predefined set of appropriate quantization step sizes. For example,a smaller quantization parameter may correspond to finer quantization (asmaller quantization step size) and a larger quantization parameter maycorrespond to coarser quantization (a larger quantization step size), orvice versa. The quantization may comprise division by a quantizationstep size and corresponding quantization or inverse quantization, forexample, performed by the inverse quantization unit 210, or may comprisemultiplication by a quantization step size. In embodiments according tosome standards such as HEVC, a quantization parameter may be used todetermine the quantization step size. Generally, the quantization stepsize may be calculated based on a quantization parameter by using afixed point approximation of an equation including division. Additionalscale factors may be introduced for quantization and dequantization torestore the norm of the residual block, where the norm of the residualblock may be modified because of a scale used in the fixed pointapproximation of the equation for the quantization step size and thequantization parameter. In an example implementation, a scale of theinverse transform may be combined with a scale of the dequantization.Alternatively, a customized quantization table may be used and signaledfrom an encoder to a decoder, for example, in a bitstream. Thequantization is a lossy operation, where the loss increases withincreasing of the quantization step size.

The inverse quantization unit 210 is configured to apply the inversequantization of the quantization unit 208 to a quantized coefficient toobtain a dequantized coefficient 211, for example, apply, based on or byusing a same quantization step size as the quantization unit 208, theinverse of a quantization scheme applied by the quantization unit 208.The dequantized coefficient 211 may also be referred to as a dequantizedresidual coefficient 211, and correspond to the transform coefficient207, although the dequantized coefficient 211 is usually different fromthe transform coefficient due to a loss caused by quantization.

The inverse transform processing unit 212 is configured to apply aninverse transform of the transform applied by the transform processingunit 206, for example, an inverse discrete cosine transform (DCT) or aninverse discrete sine transform (DST), to obtain an inverse transformblock 213 in the sample domain. The inverse transform block 213 may alsobe referred to as an inverse transform dequantized block 213 or aninverse transform residual block 213.

The reconstruction unit 214 (for example, a summator 214) is configuredto add the inverse transform block 213 (namely, the reconstructedresidual block 213) to the prediction block 265, for example, by addingsample values of the reconstructed residual block 213 and the samplevalues of the prediction block 265, to obtain a reconstructed block 215in the sample domain.

Optionally, a buffer unit 216 (“buffer” 216 for short) of, for example,the line buffer 216, is configured to buffer or store the reconstructedblock 215 and a corresponding sample value, for example, for intraprediction. In other embodiments, the encoder may be configured to usean unfiltered reconstructed block and/or a corresponding sample valuethat are stored in the buffer unit 216 for performing any type ofestimation and/or prediction, for example, intra prediction.

For example, in an embodiment, the encoder 20 may be configured so thatthe buffer unit 216 is configured to store the reconstructed block 215not only used for the intra prediction unit 254 but also used for theloop filter unit 220 (which is not shown in FIG. 2), and/or so that, forexample, the buffer unit 216 and the decoded picture buffer unit 230form one buffer. In other embodiments, a filtered block 221 and/or ablock or sample (which is not shown in FIG. 2) from the decoded picturebuffer 230 is used as an input or a basis for the intra prediction unit254.

The loop filter unit 220 (“loop filter” 220 for short) is configured tofilter the reconstructed block 215 to obtain a filtered block 221, tosmooth pixel transitions or improve video quality. The loop filter unit220 is intended to represent one or more loop filters such as adeblocking filter, a sample-adaptive offset (SAO) filter, or otherfilters, for example, a bilateral filter, an adaptive loop filter (ALF),a sharpening or smoothing filter, or a collaborative filter. Althoughthe loop filter unit 220 is shown as an in-loop filter in FIG. 2, inanother implementation, the loop filter unit 220 may be implemented as apost-loop filter. The filtered block 221 may also be referred to as afiltered reconstructed block 221. The decoded picture buffer 230 maystore a reconstructed encoded block after the loop filter unit 220performs a filtering operation on the reconstructed encoded block.

In an embodiment, the encoder 20 (correspondingly, the loop filter unit220) may be configured to output a loop filter parameter (for example,sample adaptive offset information), for example, directly or afterentropy encoding performed by the entropy encoding unit 270 or any otherentropy encoding unit, so that the decoder 30 can receive and apply thesame loop filter parameter for decoding.

The decoded picture buffer (DPB) 230 may be a reference picture memorythat stores reference picture data for use in video data encoding by theencoder 20. The DPB 230 may be formed by any one of a variety of storagedevices such as a dynamic random access memory (DRAM) (including asynchronous DRAM (SDRAM), a magnetoresistive RAM (MRAM), a resistive RAM(RRAM)), or other types of storage devices. The DPB 230 and the buffer216 may be provided by a same storage device or separate storagedevices. In an example, the decoded picture buffer (DPB) 230 isconfigured to store the filtered block 221. The decoded picture buffer230 may be further configured to store other previously filtered blocks,for example, previously reconstructed and filtered blocks 221, of thesame current picture or of different pictures, for example, previouslyreconstructed pictures, and may provide complete previouslyreconstructed, that is, decoded pictures (and corresponding referenceblocks and samples) and/or a partially reconstructed current picture(and corresponding reference blocks and samples), for example, for interprediction. In an example, if the reconstructed block 215 isreconstructed without in-loop filtering, the decoded picture buffer(DPB) 230 is configured to store the reconstructed block 215.

The prediction processing unit 260, also referred to as a blockprediction processing unit 260, is configured to receive or obtain thepicture block 203 (a current picture block 203 of the current picture201) and reconstructed picture data, for example, reference samples ofthe same (current) picture from the buffer 216 and/or reference picturedata 231 of one or more previously decoded pictures from the decodedpicture buffer 230, and process such data for prediction, namely, toprovide the prediction block 265 that may be an inter prediction block245 or an intra prediction block 255.

The mode selection unit 262 may be configured to select a predictionmode (for example, an intra or inter prediction mode) and/or acorresponding prediction block 245 or 255 to be used as the predictionblock 265, for calculation of the residual block 205 and forreconstruction of the reconstructed block 215.

In an embodiment, the mode selection unit 262 may be configured toselect the prediction mode (for example, from prediction modes supportedby the prediction processing unit 260), where the prediction modeprovides a best match or in other words a minimum residual (the minimumresidual means better compression for transmission or storage), orprovides minimum signaling overheads (the minimum signaling overheadsmean better compression for transmission or storage), or considers orbalances both. The mode selection unit 262 may be configured todetermine the prediction mode based on rate-distortion optimization(rate distortion optimization, RDO), that is, select a prediction modethat provides a minimum rate distortion or select a prediction mode forwhich related rate distortion at least satisfies a prediction modeselection criterion.

The following describes in detail prediction processing (for example,performed by the prediction processing unit 260) and mode selection (forexample, performed by the mode selection unit 262) that are performed byan example of the encoder 20.

As described above, the encoder 20 is configured to determine or selectthe best or optimal prediction mode from a set of (pre-determined)prediction modes. The set of prediction modes may comprise, for example,an intra prediction mode and/or an inter prediction mode.

A set of intra prediction modes may comprise 35 different intraprediction modes, for example, non-directional modes such as a DC (oraverage) mode and a planar mode, or directional modes such as thosedefined in H.265, or may comprise 67 different intra prediction modes,for example, non-directional modes such as a DC (or average) mode and aplanar mode, or directional modes such as those defined in H.266 underdevelopment.

In a possible implementation, a set of inter prediction modes depends onavailable reference pictures (namely, for example, at least partiallydecoded pictures stored in the DBP 230, as described above) and otherinter prediction parameters, for example, depends on whether an entirereference picture or only a part of a reference picture, for example, asearch window area around an area of a current block, is used to searchfor a best matching reference block, and/or for example, depends onwhether pixel interpolation such as half-pel and/or quarter-pelinterpolation is applied. The set of inter prediction modes maycomprise, for example, an advanced motion vector prediction (AMVP) modeand a merge mode. In a specific implementation, the set of interprediction modes may comprise a refined control point-based AMVP modeand a refined control point-based merge mode in the embodiments of thisapplication. In an example, the intra prediction unit 254 may beconfigured to perform any combination of inter prediction technologiesdescribed below.

In addition to the foregoing prediction modes, a skip mode and/or adirect mode may also be applied in the embodiments of this application.

The prediction processing unit 260 may be further configured topartition the picture block 203 into smaller block partitions orsubblocks, for example, by iteratively using quadtree (QT) partitioning,binary tree (BT) partitioning, ternary tree (TT) partitioning, or anycombination thereof, and perform, for example, prediction on each of theblock partitions or subblocks. Mode selection comprises selection of atree structure of the partitioned picture block 203 and selection of aprediction mode used for each of the block partitions or subblocks.

The inter prediction unit 244 may comprise a motion estimation (motionestimation, ME) unit (which is not shown in FIG. 2) and a motioncompensation (MC) unit (which is not shown in FIG. 2). The motionestimation unit is configured to receive or obtain a picture block 203(the current picture block 203 of the current picture 201) and a decodedpicture 231, or at least one or more previously reconstructed blocks,for example, one or more reconstructed blocks of other/differentpreviously decoded pictures 231, for motion estimation. For example, avideo sequence may comprise the current picture and the previouslydecoded picture 31. In other words, the current picture and thepreviously decoded picture 31 may be a part of or form a sequence ofpictures forming a video sequence.

For example, the encoder 20 may be configured to select a referenceblock from a plurality of reference blocks of a same picture ordifferent pictures of a plurality of other pictures, and provide areference picture and/or an offset (a spatial offset) between a location(X, Y coordinates) of the reference block and a location of the currentblock as an inter prediction parameter to the motion estimation unit(which is not shown in FIG. 2). This offset is also called a motionvector (MV).

The motion compensation unit is configured to obtain an inter predictionparameter, and perform inter prediction based on or by using the interprediction parameter to obtain an inter prediction block 245. Motioncompensation performed by the motion compensation unit (which is notshown in FIG. 2) may comprise fetching or generating the predictionblock based on a motion/block vector determined through motionestimation (possibly performing interpolation in sub-pixel precision).Interpolation filtering may generate additional pixel samples from knownpixel samples. This potentially increases a quantity of candidateprediction blocks that may be used to code a picture block. Uponreceiving a motion vector for a PU of the current picture block, themotion compensation unit 246 may locate a prediction block to which themotion vector points in one of the reference picture lists. The motioncompensation unit 246 may also generate a syntax element associated witha block and a video slice, so that the decoder 30 uses the syntaxelement to decode the picture block in the video slice.

Specifically, the inter prediction unit 244 may transmit a syntaxelement to the entropy encoding unit 270. The syntax element comprisesthe inter prediction parameter (such as indication information ofselection of an inter prediction mode used for prediction of the currentblock after a plurality of inter prediction modes are traversed). In apossible application scenario, if there is only one inter predictionmode, the inter prediction parameter may not be carried in the syntaxelement. In this case, the decoder side 30 may directly perform decodingby using a default prediction mode. It may be understood that the interprediction unit 244 may be configured to perform any combination ofinter prediction technologies.

The intra prediction unit 254 is configured to obtain, for example,receive, a picture block 203 (the current picture block) and one or morepreviously reconstructed blocks, for example, reconstructed neighboringblocks, of a same picture for intra estimation. For example, the encoder20 may be configured to select an intra prediction mode from a pluralityof (predetermined) intra prediction modes.

In an embodiment, the encoder 20 may be configured to select the intraprediction mode according to an optimization criterion, for example,based on a minimum residual (for example, an intra prediction modeproviding the prediction block 255 that is most similar to the currentpicture block 203) or minimum rate distortion.

The intra prediction unit 254 is further configured to determine theintra prediction block 255 based on, for example, an intra predictionparameter in the selected intra prediction mode. In any case, afterselecting an intra prediction mode for a block, the intra predictionunit 254 is further configured to provide an intra prediction parameter,that is, information indicating the selected intra prediction mode forthe block, to the entropy encoding unit 270. In an example, the intraprediction unit 254 may be configured to perform any combination ofintra prediction technologies.

Specifically, the intra prediction unit 254 may transmit the syntaxelement to the entropy encoding unit 270. The syntax element comprisesthe intra prediction parameter (such as indication information ofselection of an intra prediction mode used for prediction of the currentblock after a plurality of intra prediction modes are traversed). In apossible application scenario, if there is only one intra predictionmode, the intra prediction parameter may not be carried in the syntaxelement. In this case, the decoder side 30 may directly perform decodingby using a default prediction mode.

The entropy encoding unit 270 is configured to apply (or not apply) anentropy encoding algorithm or scheme (for example, a variable lengthcoding (VLC) scheme, a context-adaptive VLC (CAVLC) scheme, anarithmetic coding scheme, a context-adaptive binary arithmetic coding(CABAC) scheme, a syntax-based context-adaptive binary arithmetic coding(SBAC) scheme, a probability interval partitioning entropy (PIPE) codingscheme, or another entropy coding methodology or technology) to one orall of the quantized residual coefficient 209, the inter predictionparameter, the intra prediction parameter, and/or the loop filterparameter, to obtain encoded picture data 21 that may be output via anoutput 272, for example, in a form of an encoded bitstream 21. Theencoded bitstream may be transmitted to the video decoder 30, or storedfor subsequent transmission or retrieval by the video decoder 30. Theentropy encoding unit 270 may be further configured to entropy encodeanother syntax element for a current video slice that is being encoded.

Other structural variations of the video encoder 20 may be used toencode a video stream. For example, the non-transform based encoder 20can quantize a residual signal directly without the transform processingunit 206 for some blocks or frames. In another implementation, theencoder 20 can have the quantization unit 208 and the inversequantization unit 210 combined into a single unit.

Specifically, in this embodiment of this application, the encoder 20 maybe configured to implement an inter prediction method described in thefollowing embodiments.

It should be understood that other structural variations of the videoencoder 20 may be used to encode a video stream. For example, for somepicture blocks or picture frames, the video encoder 20 may directlyquantize a residual signal. In this case, processing by the transformprocessing unit 206 is not required, and correspondingly, processing bythe inverse transform processing unit 212 is not required either.Alternatively, for some picture blocks or picture frames, the videoencoder 20 does not generate residual data. Correspondingly, in thiscase, processing by the transform processing unit 206, the quantizationunit 208, the inverse quantization unit 210, and the inverse transformprocessing unit 212 is not required. Alternatively, the video encoder 20may directly store a reconstructed picture block as a reference block.In this case, processing by the filter 220 is not required.Alternatively, the quantization unit 208 and the inverse quantizationunit 210 in the video encoder 20 may be combined. The loop filter 220 isoptional. In addition, in a case of lossless compression coding, thetransform processing unit 206, the quantization unit 208, the inversequantization unit 210, and the inverse transform processing unit 212 arealso optional. It should be understood that, in different applicationscenarios, the inter prediction unit 244 and the intra prediction unit254 may be used selectively.

FIG. 3 is a schematic/conceptual block diagram of an example of adecoder 30 for implementing embodiments of this application. The videodecoder 30 is configured to receive, for example, encoded picture data(for example, an encoded bitstream) 21 obtained through encoding by anencoder 20, to obtain a decoded picture 231. In a decoding process, thevideo decoder 30 receives, from the video encoder 20, video data, forexample, an encoded video bitstream that represents a picture block inan encoded video slice and an associated syntax element.

In the example in FIG. 3, the decoder 30 comprises an entropy decodingunit 304, an inverse quantization unit 310, an inverse transformprocessing unit 312, a reconstruction unit 314 (for example, a summator314), a buffer 316, a loop filter 320, a decoded picture buffer 330, anda prediction processing unit 360. The prediction processing unit 360 maycomprise an inter prediction unit 344, an intra prediction unit 354, anda mode selection unit 362. In some examples, the video decoder 30 mayperform a decoding pass that is generally inverse to an encoding passdescribed with reference to the video encoder 20 in FIG. 2.

The entropy decoding unit 304 is configured to entropy decode theencoded picture data 21 to obtain, for example, a quantized coefficient309 and/or a decoded encoding parameter (which is not shown in FIG. 3),for example, any one or all of an inter prediction parameter, an intraprediction parameter, a loop filter parameter, and/or another syntaxelement (that are decoded). The entropy decoding unit 304 is furtherconfigured to forward the inter prediction parameter, the intraprediction parameter, and/or the another syntax element to theprediction processing unit 360. The video decoder 30 may receive asyntax element at a video slice level and/or a picture block level.

The inverse quantization unit 310 may be identical in function to theinverse quantization unit 110, the inverse transform processing unit 312may be identical in function to the inverse transform processing unit212, the reconstruction unit 314 may be identical in function to thereconstruction unit 214, the buffer 316 may be identical in function tothe buffer 216, the loop filter 320 may be identical in function to theloop filter 220, and the decoded picture buffer 330 may be identical infunction to the decoded picture buffer 230.

The prediction processing unit 360 may comprise the inter predictionunit 344 and the intra prediction unit 354. The inter prediction unit344 may be similar in function to the inter prediction unit 244, and theintra prediction unit 354 may be similar in function to the intraprediction unit 254. The prediction processing unit 360 is usuallyconfigured to perform block prediction and/or obtain a prediction block365 from the encoded data 21, and receive or obtain (explicitly orimplicitly) a prediction-related parameter and/or information about aselected prediction mode, for example, from the entropy decoding unit304.

When the video slice is encoded into an intra encoded (I) slice, theintra prediction unit 354 in the prediction processing unit 360 isconfigured to generate a prediction block 365 of a picture block in thecurrent video slice based on a signaled intra prediction mode and dataof a previously decoded block of a current frame or picture. When thevideo frame is encoded into an inter encoded (namely, B or P) slice, theinter prediction unit 344 (for example, a motion compensation unit) inthe prediction processing unit 360 is configured to generate aprediction block 365 of a video block in the current video slice basedon a motion vector and the another syntax element that is received fromthe entropy decoding unit 304. In inter prediction, a prediction blockmay be generated from a reference picture in a reference picture list.The video decoder 30 may construct reference frame lists, a list 0 and alist 1, by using a default construction technology and based onreference pictures stored in the DPB 330.

The prediction processing unit 360 is configured to determine predictioninformation of the video block in the current video slice by parsing themotion vector and the another syntax element, and generate, by using theprediction information, the prediction block of the current video blockthat is being decoded. In an example embodiment of this application, theprediction processing unit 360 determines, by using some received syntaxelements, a prediction mode (for example, intra prediction or interprediction) for encoding the video block in the video slice, an interprediction slice type (for example, a B slice, a P slice, or a GPBslice), construction information of one or more of the reference picturelists for the slice, a motion vector of each inter encoded video blockin the slice, an inter prediction status of each inter encoded videoblock in the slice, and other information, to decode the video block inthe current video slice. In another example of this disclosure, thesyntax element received by the video decoder 30 from the bitstreamcomprises a syntax element in one or more of an adaptive parameter set(APS), a sequence parameter set (SPS), a picture parameter set (PPS), ora slice header.

The inverse quantization unit 310 may be configured to perform inversequantization (namely, dequantization) on a quantized transformcoefficient provided in the bitstream and decoded by the entropydecoding unit 304. An inverse quantization process may comprise use of aquantization parameter calculated by the video encoder 20 for each videoblock in the video slice, to determine a degree of quantization thatshould be applied and likewise, a degree of inverse quantization thatshould be applied.

The inverse transform processing unit 312 is configured to apply aninverse transform (for example, an inverse DCT, an inverse integertransform, or a conceptually similar inverse transform process) to atransform coefficient, to generate a residual block in a pixel domain.

The reconstruction unit 314 (for example, the summator 314) isconfigured to add an inverse transform block 313 (namely, areconstructed residual block 313) to the prediction block 365, forexample, by adding sample values of the reconstructed residual block 313and sample values of the prediction block 365, to obtain a reconstructedblock 315 in a sample domain.

The loop filter unit 320 (either in a coding loop or after a codingloop) is configured to filter the reconstructed block 315 to obtain afiltered block 321, to smooth pixel transitions or improve videoquality. In an example, the loop filter unit 320 may be configured toperform any combination of filtering technologies described below. Theloop filter unit 320 is intended to represent one or more loop filterssuch as a deblocking filter, a sample-adaptive offset (SAO) filter, orother filters, for example, a bilateral filter, an adaptive loop filter(ALF), a sharpening or smoothing filter, or a collaborative filter.Although the loop filter unit 320 is shown as an in-loop filter in FIG.3, in another implementation, the loop filter unit 320 may beimplemented as a post-loop filter.

Then, a decoded video block 321 in a given frame or picture is stored inthe decoded picture buffer 330 that stores a reference picture used forsubsequent motion compensation.

The decoder 30 is configured to output, for example, the decoded picture31 via an output 332, for present or viewing to a user.

Other variations of the video decoder 30 may be used to decode acompressed bitstream. For example, the decoder 30 can generate an outputvideo stream without the loop filter unit 320. For example, thenon-transform based decoder 30 can inverse-quantize a residual signaldirectly without the inverse transform processing unit 312 for someblocks or frames. In another implementation, the video decoder 30 mayhave the inverse quantization unit 310 and the inverse transformprocessing unit 312 combined into a single unit.

Specifically, in this embodiment of this application, the decoder 30 maybe configured to implement an inter prediction method described in thefollowing embodiments.

It should be understood that other structural variations of the videodecoder 30 may be used to decode an encoded video bitstream. Forexample, the video decoder 30 may generate an output video streamwithout processing by the filter 320. Alternatively, for some pictureblocks or picture frames, the entropy decoding unit 304 in the videodecoder 30 does not obtain a quantized coefficient through decoding.Correspondingly, in this case, processing by the inverse quantizationunit 310 and the inverse transform processing unit 312 is not required.The loop filter 320 is optional. In addition, in a case of losslesscompression, the inverse quantization unit 310 and the inverse transformprocessing unit 312 are also optional. It should be understood that, indifferent application scenarios, the inter prediction unit and the intraprediction unit may be used selectively.

It should be understood that, in the encoder 20 and the decoder 30 inthis application, a processing result of an operation may be furtherprocessed and then output to a next operation. For example, after anoperation such as interpolation filtering, motion vector derivation, orloop filtering, a further operation, such as clip or shift, is performedon a processing result of the corresponding operation.

For example, a motion vector that is of a control point of a currentpicture block and that is derived based on a motion vector of aneighboring affine coded block may be further processed. This is notlimited in this application. For example, a value of the motion vectoris constrained to be within a specific bit width range. Assuming that anallowed bit width of the motion vector is bitDepth, the value of themotion vector ranges from −2{circumflex over ( )}(bitDepth−1) to2{circumflex over ( )}(bitDepth−1)−1, where the symbol “{circumflex over( )}” represents exponentiation. If bitDepth is 16, the value rangesfrom −32768 to 32767. If bitDepth is 18, the value ranges from −131072to 131071. The value of the motion vector may be constrained in eitherof the following two manners:

Manner 1: An overflow most significant bit of the motion vector isremoved:

ux=(vx+2^(bitDepth))%2^(bitDepth)

vx=(ux>=2^(bitDepth)−1)?(ux−2^(bitDepth)):ux

uy=(vy+2^(bitDepth))%2^(bitDepth)

vy=(uy>=2^(bitDepth)−1)?(uy−2^(bitDepth)):uy

For example, a value of vx is −32769, and 32767 is derived according tothe foregoing formulas. A value is stored on a computer in a two'scomplement representation, a two's complement representation of −32769is 1,0111,1111,1111,1111 (17 bits), and processing performed by thecomputer for overflowing is discarding a most significant bit.Therefore, a value of vx is 0111,1111,1111,1111, that is, 32767. Thisvalue is consistent with the result derived through processing accordingto the formulas.

Manner 2: Clipping is performed on the motion vector, and the followingformulas are used:

vx=Clip3(−2^(bitDepth−1),2^(bitDepth−1)−1,vx)

vy=Clip3(−2^(bitDepth−1),2^(bitDepth−1)−1,vy)

In the foregoing formulas, Clip3 is defined as clipping a value of z toa range [x, y].

${Clip3\left( {x,y,z} \right)} = \left\{ \begin{matrix}{x;} & {\ {z < x}} \\{{y;}\ } & {z > y} \\{{z;}\ } & {otherwise}\end{matrix} \right.$

FIG. 4 is a schematic structural diagram of a video coding device 400(for example, a video encoding device 400 or a video decoding device400) according to an embodiment of this application. The video codingdevice 400 is suitable for implementing the embodiments described inthis specification. In an embodiment, the video coding device 400 may bea video decoder (for example, the decoder 30 in FIG. 1A) or a videoencoder (for example, the encoder 20 in FIG. 1A). In another embodiment,the video coding device 400 may be one or more components of the decoder30 in FIG. 1A or the encoder 20 in FIG. 1A.

The video coding device 400 comprises: ingress ports 410 and a receiver(Rx) 420 that are configured to receive data; a processor, a logic unit,or a central processing unit (CPU) 430 that is configured to processdata; a transmitter (Tx) 440 and egress ports 450 that are configured totransmit data; and a memory 460 configured to store data. The videocoding device 400 may further comprise optical-to-electrical componentsand electrical-to-optical (EO) components that are coupled to theingress ports 410, the receiver 420, the transmitter 440, and the egressports 450, for egress or ingress of an optical or electrical signal.

The processor 430 is implemented by hardware and software. The processor430 may be implemented as one or more CPU chips, cores (for example, amulti-core processor), FPGAs, ASICs, and DSPs. The processor 430communicates with the ingress ports 410, the receiver 420, thetransmitter 440, the egress ports 450, and the memory 460. The processor430 comprises a coding module 470 (for example, an encoding module 470or a decoding module 470). The encoding/decoding module 470 implementsthe embodiments disclosed in this specification, to implement a chromablock prediction method provided in the embodiments of this application.For example, the encoding/decoding module 470 implements, processes, orprovides various coding operations. Therefore, the encoding/decodingmodule 470 provides a substantial improvement to a function of the videocoding device 400, and affects transformation of the video coding device400 to a different state. Alternatively, the encoding/decoding module470 is implemented as an instruction that is stored in the memory 460and executed by the processor 430.

The memory 460 comprises one or more disks, tape drives, and solid-statedrives, and may be used as an overflow data storage device, to storeprograms when such programs are selected for execution, and to store aninstruction and data that are read during program execution. The memory460 may be volatile and/or nonvolatile, and may be a read-only memory(ROM), a random access memory (RAM), a ternary content-addressablememory (TCAM), and/or a static random access memory (SRAM).

FIG. 5 is a simplified block diagram of an apparatus 500 that may beused as either or both of the source device 12 and the destinationdevice 14 in FIG. 1A according to an example embodiment. The apparatus500 can implement the technologies of this application. In other words,FIG. 5 is a schematic block diagram of an implementation of an encodingdevice or a decoding device (referred to as a coding device 500 forshort) according to an embodiment of this application. The coding device500 may comprise a processor 510, a memory 530, and a bus system 550.The processor and the memory are connected through the bus system. Thememory is configured to store an instruction. The processor isconfigured to execute the instruction stored in the memory. The memoryof the coding device stores program code. The processor may invoke theprogram code stored in the memory to perform various video encoding ordecoding methods described in embodiments of this application. To avoidrepetition, details are not described herein.

In this embodiment of this application, the processor 510 may be acentral processing unit (“CPU” for short). Alternatively, the processor510 may be another general-purpose processor, a digital signal processor(DSP), an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA) or another programmable logicdevice, a discrete gate or transistor logic device, a discrete hardwarecomponent, or the like. The general-purpose processor may be amicroprocessor, any conventional processor, or the like.

The memory 530 may comprise a read-only memory (ROM) device or a randomaccess memory (RAM) device. Any other suitable type of storage devicemay alternatively be used as the memory 530. The memory 530 may comprisecode and data 531 accessed by the processor 510 through the bus 550. Thememory 530 may further comprise an operating system 533 and anapplication program 535. The application program 535 comprises at leastone program that allows the processor 510 to perform a video encoding ordecoding method described embodiments of in this application. Forexample, the application programs 535 may comprise applications 1 to N.The applications further comprise a video encoding or decodingapplication (referred to as a video coding application for short) thatperforms the video encoding or decoding method described in embodimentsof this application.

The bus system 550 may not only comprise a data bus, but also comprise apower bus, a control bus, a status signal bus, and the like. However,for clear description, various types of buses in the figure are markedas the bus system 550.

Optionally, the coding device 500 may further comprise one or moreoutput devices, for example, a display 570. In an example, the display570 may be a touch sensitive display that combines a display and a touchsensitive unit that is operable to sense a touch input. The display 570may be connected to the processor 510 through the bus 550.

The following first describes concepts in embodiments of thisapplication.

1. Inter Prediction Mode:

In HEVC, two inter prediction modes are used: an advanced motion vectorprediction (AMVP) mode and a merge mode.

In the AMVP mode, spatial or temporal neighboring encoded blocks(denoted as adjacent blocks) of a current block are first traversed; acandidate motion vector list (which may also be referred to as a motioninformation candidate list) is constructed based on motion informationof the neighboring blocks; and then an optimal motion vector isdetermined from the candidate motion vector list based on arate-distortion cost. Candidate motion information corresponding to aminimum rate-distortion cost is used as a motion vector predictor (MVP)of the current block. Both locations of the neighboring blocks and atraversal order thereof are predefined. The rate-distortion cost iscalculated according to Formula (1), where J represents therate-distortion cost (RD cost), SAD is a sum of absolute differences(SAD) between original sample values and prediction sample valuesobtained through motion estimation by using a candidate motion vectorpredictor, R represents a bit rate, and represents a Lagrangemultiplier. An encoder side transfers an index value of the selectedmotion vector predictor in the candidate motion vector list and areference frame index value to a decoder side. Further, motion search isperformed in a neighborhood centered on the MVP, to obtain an actualmotion vector of the current block. The encoder side transfers adifference (motion vector difference) between the MVP and the actualmotion vector to the decoder side.

J=SAD+λR  (1)

In the merge mode, a candidate motion vector list is first constructedbased on motion information of spatial or temporal neighboring encodedblocks of a current block. Then, optimal motion information isdetermined from the candidate motion vector list as motion informationof the current block based on a rate-distortion cost. An index value(denoted as a merge index hereinafter) of a location of the optimalmotion information in the candidate motion vector list is transferred toa decoder side. Spatial and temporal candidate motion information of thecurrent block is shown in FIG. 6A. The spatial candidate motioninformation is from five spatial neighboring blocks (A0, A1, B0, B1, andB2). If a neighboring block is unavailable (the neighboring block doesnot exist, or the neighboring block is not encoded, or a prediction modeused for the neighboring block is not an inter prediction mode), motioninformation of the neighboring block is not added to the candidatemotion vector list. The temporal candidate motion information of thecurrent block is obtained by scaling an MV of a block at a correspondinglocation in a reference frame based on picture order counts (POC) of thereference frame and a current frame. Whether a block at a location T inthe reference frame is available is first determined. If the block isunavailable, a block at a location C is selected.

Similar to the AMVP mode, in the merge mode, both locations of theneighboring blocks and a traversal order thereof are also predefined. Inaddition, the locations of the neighboring blocks and the transversalorder thereof may be different in different modes.

It can be learned that one candidate motion vector list needs to bemaintained in both the AMVP mode and the merge mode. Before new motioninformation is added to the candidate list each time, whether samemotion information exists in the list is first checked. If the samemotion information exists in the list, the motion information is notadded to the list. This checking process is referred to as pruning ofthe candidate motion vector list. Pruning of the list is to avoidexistence of the same motion information in the list, to avoid redundantrate-distortion cost calculation.

In inter prediction in HEVC, same motion information is used for allsamples of a coding block, and then motion compensation is performedbased on the motion information, to obtain predictors of the samples ofthe coding block. In the coding block, however, not all samples havesame motion features. Using the same motion information may result ininaccurate motion compensation prediction and more residual information.

In existing video coding standards, translational motion model basedblock matching motion estimation is used, and it is assumed that motionof all samples in a block is consistent. However, in the real world,there are a variety of motion. Many objects are in non-translationalmotion, for example, a rotating object, a roller coaster spinning indifferent directions, a display of fireworks, and some stunts in movies,especially moving object in a UGC scenario. For these moving objects, ifa translational motion model based block motion compensation technologyin the existing coding standards is used for coding, coding efficiencymay be greatly affected. In view of this, a non-translational motionmodel, for example, an affine motion model, is introduced to improve thecoding efficiency.

Based on this, in terms of different motion models, the AMVP mode may beclassified into a translational model based AMVP mode and anon-translational model based AMVP mode, and the merge mode may beclassified into a translational model based merge mode and anon-translational model based merge mode.

2. Non-Translational Motion Model:

Non-translational motion model based prediction means that a same motionmodel is used on both encoder and decoder sides to derive motioninformation of each subblock of a current block, and motion compensationis performed based on the motion information of the subblock to obtain aprediction block. In this way, prediction efficiency is improved. Commonnon-translational motion models comprise a 4-parameter affine motionmodel and a 6-parameter affine motion model.

A subblock in the embodiments of this application may be a sample or anN₁×N₂ sample block obtained by using a particular partitioning method,where both N₁ and N₂ are positive integers, and N₁ may be equal to N₂ ormay not be equal to N₂.

The 4-parameter affine motion model is expressed according to Formula(2):

$\begin{matrix}\left\{ \begin{matrix}{{vx} = {a_{1} + {a_{3}x} + {a_{4}y}}} \\{{vy} = {a_{2} - {a_{4}x} + {a_{3}y}}}\end{matrix} \right. & (2)\end{matrix}$

The 4-parameter affine motion model may be represented by motion vectorsof two samples and their coordinates relative to the top-left sample ofthe current block. A sample used for representing a motion modelparameter is referred to as a control point. If the top-left sample (0,0) and the top-right sample (W, 0) are used as control points,respective motion vectors (vx₀, vy₀) and (vx₁, vy₁) of the top-leftvertex and the top-right vertex of the current block are firstdetermined. Then, motion information of each subblock of the currentblock is obtained according to Formula (3), where (x, y) representscoordinates of the subblock relative to the top-left sample of thecurrent block, and W represents the width of the current block.

$\begin{matrix}\left\{ \begin{matrix}{{vx} = {{\frac{{vx_{1}} - {vx_{0}}}{W}x} - {\frac{{vy_{1}} - {vy_{0}}}{W}y} + {vx_{0}}}} \\{{vy} = {{\frac{{vy_{1}} - {vy_{0}}}{W}x} + {\frac{{vx_{1}} - {vx_{0}}}{W}y} + {vy_{0}}}}\end{matrix} \right. & (3)\end{matrix}$

The 6-parameter affine motion model is expressed according to Formula(4):

$\begin{matrix}\left\{ \begin{matrix}{{vx} = {a_{1} + {a_{3}x} + {a_{4}y}}} \\{{vy} = {a_{2} + {a_{5}x} + {a_{6}y}}}\end{matrix} \right. & (4)\end{matrix}$

The 6-parameter affine motion model may be represented by motion vectorsof three samples and their coordinates relative to the top-left sampleof the current block. If the top-left sample (0, 0), the top-rightsample (W, 0), and the bottom-left sample (0, H) of the current blockare used as control points, respective motion vectors (vx₀, vy₀), (vx₁,vy₁), and (vx₂, vy₂) of the top-left control point, the top-rightcontrol point, and the bottom-left control point of the current blockare first determined. Then, motion information of each subblock of thecurrent block is obtained according to Formula (5), where (x, y)represents coordinates of the subblock relative to the top-left sampleof the current block, and W and H represent the width and the height ofthe current block, respectively.

$\begin{matrix}\left\{ \begin{matrix}{{vx} = {{\frac{{vx}_{1} - {vx}_{0}}{W}x} + {\frac{{vx}_{2} - {vy}_{0}}{H}y} + {vx}_{0}}} \\{{vy} = {{\frac{{vy}_{1} - {vy}_{0}}{W}x} + {\frac{{vy}_{2} - {vx}_{0}}{H}y} + {vy}_{0}}}\end{matrix} \right. & (5)\end{matrix}$

A coding block that is predicted by using an affine motion model isreferred to as an affine coded block.

Generally, motion information of a control point of an affine codedblock may be obtained by using an affine motion model based advancedmotion vector prediction (AMVP) mode or an affine motion model basedmerge mode.

The motion information of the control point of the current coding blockmay be obtained by using an inherited control point motion vectorprediction method or a constructed control point motion vectorprediction method.

3. Inherited Control Point Motion Vector Prediction Method:

The inherited control point motion vector prediction method is to use amotion model of a neighboring encoded affine coded block to determinecandidate control point motion vectors of a current block.

A current block shown in FIG. 6B is used as an example. Blocks atneighboring locations around the current block are traversed in aspecified order, for example, A1->B1->B0->A0->B2, to find an affinecoded block including a block at a neighboring location of the currentblock, and to obtain control point motion information of the affinecoded block. Further, a control point motion vector (used in the mergemode) or a control point motion vector predictor (used in the AMVP mode)of the current block is derived by using a motion model constructedbased on the control point motion information of the affine coded block.The order A1->B1->B0->A0->B2 is merely used as an example. Anothercombination order may also be used in to this application. In addition,the blocks at the neighboring locations are not limited to A1, B1, B0,A0, and B2.

The block at the neighboring location may be a sample or a sample blockof a preset size obtained by using a particular partitioning method. Forexample, the sample block may be a 4×4 sample block, a 4×2 sample block,or a sample block of another size. This is not limited herein.

The following describes a determining process by using A1 as an example,and similar processes can be employed for other cases.

As shown in FIG. 4, if a coding block in which A1 is located is a4-parameter affine coded block, a motion vector (vx₀, vy₀) of thetop-left vertex (x₄, y₄) and a motion vector (vx₅, vy₅) of the top-rightvertex (x₅, y₅) of the affine coded block are obtained. A motion vector(vx₀, vy₀) of the top-left vertex (x₀, y₀) of the current affine codedblock is calculated according to Formula (6), and a motion vector (vx₁,vy₁) of the top-right vertex (x₁, y₁) of the current affine coded blockis calculated according to Formula (7).

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{0} = {{vx}_{4} + {\frac{\left( {{vx}_{5} - {vx}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{0} - x_{4}} \right)} - {\frac{\left( {{vy}_{5} - {vy}_{4}} \right)}{x_{5} - x_{4}} \times \left( {y_{0} - y_{4}} \right)}}} \\{{vy}_{0} = {{vy}_{4} + {\frac{\left( {{vy}_{5} - {vy}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{0} - x_{4}} \right)} + {\frac{\left( {{vx}_{5} - {vx}_{4}} \right)}{x_{5} - x_{4}} \times \left( {y_{0} - y_{4}} \right)}}}\end{matrix} \right. & (6)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx}_{1} = {{vx}_{4} + {\frac{\left( {{vx}_{5} - {vx_{4}}} \right)}{x_{5} - x_{4}} \times \left( {x_{1} - x_{4}} \right)} - {\frac{\left( {{vy}_{5} - {vy_{4}}} \right)}{x_{5} - x_{4}} \times \left( {y_{1} - y_{4}} \right)}}} \\{{vy}_{1} = {{vy}_{4} + {\frac{\left( {{vy}_{5} - {vy_{4}}} \right)}{x_{5} - x_{4}} \times \left( {x_{1} - x_{4}} \right)} + {\frac{\left( {{vx}_{5} - {vx_{4}}} \right)}{x_{5} - x_{4}} \times \left( {y_{1} - y_{4}} \right)}}}\end{matrix} \right. & (7)\end{matrix}$

A combination of the motion vector (vx₀, vy₀) of the top-left vertex(x₀, y₀) and the motion vector (vx₁, vy₁) of the top-right vertex (x₁,y₁) of the current block that are obtained based on the affine codedblock in which A1 is located is the candidate control point motionvectors of the current block.

If a coding block in which A1 is located is a 6-parameter affine codedblock, a motion vector (vx₄, vy₄) of the top-left vertex (x₄, y₄), amotion vector (vx₅, vy₅) of the top-right vertex (x₅, y₅), and a motionvector (vx₆, vy₆) of the bottom-left vertex (x₆, y₆) of the affine codedblock are obtained. A motion vector (vx₀, vy₀) of the top-left vertex(x₀, y₀) of the current block is calculated according to Formula (8), amotion vector (vx₁, vy₁) of the top-right vertex (x₁, y₁) of the currentblock is calculated according to Formula (9), and a motion vector (vx₂,vy₂) of the bottom-left vertex (x₂, y₂) of the current block iscalculated according to Formula (10).

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{0} = {{vx}_{4} + {\frac{\left( {{vx}_{5} - {vx}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{0} - x_{4}} \right)} + {\frac{\left( {{vx}_{6} - {vx}_{4}} \right)}{y_{6} - y_{4}} \times \left( {y_{0} - y_{4}} \right)}}} \\{{vy}_{0} = {{vy}_{4} + {\frac{\left( {{vy}_{5} - {vy}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{0} - x_{4}} \right)} + {\frac{\left( {{vy}_{6} - {vy}_{4}} \right)}{y_{6} - x_{4}} \times \left( {y_{0} - y_{4}} \right)}}}\end{matrix} \right. & (8)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx}_{1} = {{vx}_{4} + {\frac{\left( {{vx}_{5} - {vx}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{1} - x_{4}} \right)} + {\frac{\left( {{vx}_{6} - {vx}_{4}} \right)}{y_{6} - y_{4}} \times \left( {y_{1} - y_{4}} \right)}}} \\{{vy}_{1} = {{vy}_{4} + {\frac{\left( {{vy}_{5} - {vy}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{1} - x_{4}} \right)} + {\frac{\left( {{vy}_{6} - {vy}_{4}} \right)}{y_{6} - y_{4}} \times \left( {y_{1} - y_{4}} \right)}}}\end{matrix} \right. & (9)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx}_{2} = {{vx}_{4} + {\frac{\left( {{vx}_{5} - {vx}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{2} - x_{4}} \right)} + {\frac{\left( {{vx}_{6} - {vx}_{4}} \right)}{y_{6} - y_{4}} \times \left( {y_{2} - y_{4}} \right)}}} \\{{vy}_{2} = {{vy}_{4} + {\frac{\left( {{vy}_{5} - {vy}_{4}} \right)}{x_{5} - x_{4}} \times \left( {x_{2} - x_{4}} \right)} + {\frac{\left( {{vy}_{6} - {vy}_{4}} \right)}{y_{6} - y_{4}} \times \left( {y_{2} - y_{4}} \right)}}}\end{matrix} \right. & (10)\end{matrix}$

A combination of the motion vector (vx₀, vy₀) of the top-left vertex(x₀, y₀), the motion vector (vx₁, vy₁) of the top-right vertex (x₁, y₁),and the motion vector (vx₂, vy₂) of the bottom-left vertex (x₂, y₂) ofthe current block that are obtained based on the affine coded block inwhich A1 is located is the candidate control point motion vector of thecurrent block.

It should be noted that other motion models, candidate locations, andsearch and traversal orders may also be used in embodiments of thisapplication. Details are not described in the embodiments of thisapplication.

It should be noted that methods for representing motion models ofneighboring and current coding blocks based on other control points mayalso be used in embodiments of this application. Details are notdescribed herein.

4. Constructed Control Point Motion Vector Prediction Method 1:

The constructed control point motion vector prediction method accordingto an embodiment is to combine motion vectors of neighboring encodedblocks around a control point of a current block as a control pointmotion vector of a current affine coded block, without consideringwhether the neighboring encoded blocks are affine coded blocks.

Motion vectors of the top-left vertex and the top-right vertex of thecurrent block are determined based on motion information of theneighboring encoded blocks around the current coding block. FIG. 6C isused as an example to describe the constructed control point motionvector prediction method. It should be noted that FIG. 6C is merely usedan example.

As shown in FIG. 6C, motion vectors of neighboring encoded blocks A2,B2, and B3 at the top-left corner are used as candidate motion vectorsfor a motion vector of the top-left vertex of the current block; andmotion vectors of neighboring encoded blocks B1 and B0 at the top-rightcorner are used as candidate motion vectors for a motion vector of thetop-right vertex of the current block. The candidate motion vectors ofthe top-left vertex and the top-right vertex are combined to constitutea plurality of 2-tuples. Motion vectors of two encoded blocks comprisedin a 2-tuple may be used as candidate control point motion vectors ofthe current block, as shown in the following Formula (11A):

{v _(A2) ,v _(B1) },{v _(A2) ,v _(B0) },{v _(B2) ,v _(B1) },{v _(B2) ,v_(B0) },{v _(B3) ,v _(B1) },{v _(B3) ,v _(B0)}  (11A)

where v_(A2) represents the motion vector of A2, v_(B1) represents themotion vector of B1, v_(B0) represents the motion vector of B0, v_(B2)represents the motion vector of B2, and v_(B3) represents the motionvector of B3.

As shown in FIG. 6C, motion vectors of neighboring encoded blocks A2,B2, and B3 at the top-left corner are used as candidate motion vectorsfor a motion vector of the top-left vertex of the current block; motionvectors of neighboring encoded blocks B1 and B0 at the top-right cornerare used as candidate motion vectors for a motion vector of thetop-right vertex of the current block; and motion vectors of neighboringencoded blocks A0 and A1 at the bottom-left corner are used as candidatemotion vectors for a motion vector of the bottom-left vertex of thecurrent block. The candidate motion vectors of the top-left vertex, thetop-right vertex, and the bottom-left vertex are combined to constitute3-tuples. Motion vectors of three encoded blocks comprised in a 3-tuplemay be used as candidate control point motion vectors of the currentblock, as shown in the following Formulas (11B) and (11C):

{v _(A2) ,v _(B1) ,v _(A0) },{v _(A2) ,v _(B0) ,v _(A0) },{v _(B2) ,v_(B1) ,v _(A0) },{v _(B2) ,v _(B0) ,v _(A0) },{v _(B3) ,v _(B1) ,v _(A0)},{v _(B3) ,v _(B0) ,v _(A0)}   (11B)

{v _(A2) ,v _(B1) ,v _(A1) },{v _(A2) ,v _(B0) ,v _(A1) },{v _(B2) ,v_(B1) ,v _(A1) },{v _(B2) ,v _(B1) ,v _(A1) },{v _(B3) ,v _(B1) ,v _(A1)},{v _(B3) ,v _(B0) ,v _(A1)}   (11C)

where v_(A2) represents the motion vector of A2, v_(B1) represents themotion vector of B1, v_(B0) represents the motion vector of B0, v_(B2)represents the motion vector of B2, v_(B3) represents the motion vectorof B3, v_(A0) represents the motion vector of A0, and v_(A1) representsthe motion vector of A1.

It should be noted that other methods for combining control point motionvectors may also be used in embodiments of this application. Details arenot described herein.

It should be noted that methods for representing motion models ofneighboring and current coding blocks based on other control points mayalso be used in embodiments of this application. Details are notdescribed herein.

5. Constructed Control Point Motion Vector Prediction Method 2,According to Some Embodiments, as Shown in FIG. 6D:

Operation 601: Obtain motion information of control points of a currentblock.

For example, in FIG. 6C, CPk (k=1, 2, 3, 4) represents a kth controlpoint. A0, A1, A2, B0, B1, B2, and B3 are spatial neighboring locationsof the current block and are used to predict CP1, CP2, or CP3. T is atemporal neighboring location of the current block and is used topredict CP4.

It is assumed that coordinates of CP1, CP2, CP3, and CP4 are (0, 0), (W,0), (H, 0), and (W, H), respectively, where W and H represent the widthand the height of the current block.

For each control point, motion information of the control point isobtained in the following order:

(1) For CP1, a check order is B2->A2->B3. If B2 is available, motioninformation of B2 is used for CP1. Otherwise, A2 and B3 are checked. Ifmotion information of all the three locations is unavailable, motioninformation of CP1 cannot be obtained.

(2) For CP2, a check order is B0->B1. If B0 is available, motioninformation of B0 is used for CP2. Otherwise, B1 is checked. If motioninformation of both the locations is unavailable, motion information ofCP2 cannot be obtained.

(3) For CP3, a checking order is A0->A1.

(4) For CP4, motion information of T is used.

Herein, that X is available means that a block at X (X is A0, A1, A2,B0, B1, B2, B3 or T) is already encoded and an inter prediction mode isused for the block. Otherwise, X is unavailable.

It should be noted that other methods for obtaining control point motioninformation may also be used in embodiments of this application. Detailsare not described herein.

Operation 602: Combine the motion information of the control points toobtain constructed control point motion information.

Motion information of two control points is combined to constitute a2-tuple, to construct a 4-parameter affine motion model. Combinations oftwo control points may be {CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2,CP4}, {CP1, CP3}, and {CP3, CP4}. For example, a 4-parameter affinemotion model constructed by using a 2-tuple including the control pointsCP1 and CP2 may be denoted as Affine (CP1, CP2).

Motion information of three control points is combined to constitute a3-tuple, to construct a 6-parameter affine motion model. Combinations ofthree control points may be {CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3,CP4}, and {CP1, CP3, CP4}. For example, a 6-parameter affine motionmodel constructed by using a 3-tuple including the control points CP1,CP2, and CP3 may be denoted as Affine (CP1, CP2, CP3).

Motion information of four control points is combined to constitute a4-tuple, to construct an 8-parameter bilinear motion model. An8-parameter bilinear model constructed by using a 4-tuple including thecontrol points CP1, CP2, CP3, and CP4 may be denoted as Bilinear (CP1,CP2, CP3, CP4).

In the embodiments of this application, for ease of description, acombination of motion information of two control points (or two encodedblocks) is simply referred to as a 2-tuple, a combination of motioninformation of three control points (or three encoded blocks) is simplyreferred to as a 3-tuple, and a combination of motion information offour control points (or four encoded blocks) is simply referred to as a4-tuple.

These models are traversed in a preset order. If motion information of acontrol point corresponding to a combination model is unavailable, it isconsidered that the model is unavailable. Otherwise, a reference frameindex of the model is determined, and a motion vector of the controlpoint is scaled. If motion information of all control points afterscaling is consistent, the model is invalid. If all motion informationof control points controlling the model is available, and the model isvalid, the motion information of the control points used to constructthe model is added to a motion information candidate list.

A control point motion vector scaling method is shown in Formula (12):

$\begin{matrix}{{MV}_{s} = {\frac{{CurPoc} - {DesPoc}}{{CurPoc} - {SrcPoc}} \times {MV}}} & (12)\end{matrix}$

where CurPoc represents a POC number of a current frame, DesPocrepresents a POC number of a reference frame of a current block, SrcPocrepresents a POC number of a reference frame of a control point, MV_(s)represents a motion vector obtained after scaling, and MV represents amotion vector of a control point.

It should be noted that different combinations of control points may beconverted into control points at a same location.

For example, a 4-parameter affine motion model obtained based on acombination of {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, or {CP3,CP4} is represented by {CP1, CP2} or {CP1, CP2, CP3} after conversion. Aconversion method comprises: substituting motion vectors and coordinateinformation of control points into Formula (2) to obtain modelparameters; and then substituting coordinate information of {CP1, CP2}into Formula (3) to obtain motion vectors of {CP1, CP2}.

More directly, conversion may be performed according to Formulas (13) to(21), where W represents the width of the current block, and Hrepresents the height of the current block. In Formulas (13) to (21),(vx₀, vy₀) represents a motion vector of CP1, (vx₁, vy₁) represents amotion vector of CP2, (vx₂, vy₂) represents a motion vector of CP3, and(vx₃, vy₃) represents a motion vector of CP4.

{CP1, CP2} may be converted into {CP1, CP2, CP3} according to Formula(13). In other words, the motion vector of CP3 in {CP1, CP2, CP3} may bedetermined according to Formula (13):

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{2} = {{{- \frac{{vy}_{1} - {vy}_{0}}{W}}H} + {vx}_{0}}} \\{{vy}_{2} = {{{+ \frac{{vx}_{1} - {vx}_{0}}{W}}H} + {vy}_{0}}}\end{matrix} \right. & (13)\end{matrix}$

{CP1, CP3} may be converted to {CP1, CP2} or {CP1, CP2, CP3} accordingto Formula (14):

$\begin{matrix}\left\{ \begin{matrix}{{vx_{1}} = {{{+ \frac{{vy}_{2} - {vy}_{0}}{H}}W} + {vx}_{0}}} \\{{vy}_{1} = {{{- \frac{{vx}_{2} - {vx}_{0}}{H}}W} + {vy}_{0}}}\end{matrix} \right. & (14)\end{matrix}$

{CP2, CP3} may be converted into {CP1, CP2} or {CP1, CP2, CP3} accordingto Formula (15):

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{0} = {{\frac{{vx}_{2} - {vx}_{1}}{{W*W} + {H*H}}W*W} - {\frac{{vy}_{2} - {vy}_{1}}{{W*W} + {H*H}}H*W} + {vx}_{1}}} \\{{vy}_{0} = {{\frac{{vy}_{2} - {vy}_{1}}{{W*W} + {H*H}}W*W} + {\frac{{vx}_{2} - {vx}_{1}}{{W*W} + {H*H}}H*W} + {vy}_{1}}}\end{matrix} \right. & (15)\end{matrix}$

{CP1, CP4} may be converted into {CP1, CP2} or {CP1, CP2, CP3} accordingto Formula (16) or (17):

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{1} = {{\frac{{vx}_{3} - {vx}_{0}}{{W*W} + {H*H}}W*W} + {\frac{{vy}_{3} - {vy}_{0}}{{W*W} + {H*H}}H*W} + {vx}_{0}}} \\{{vy}_{1} = {{\frac{{vy}_{3} - {vy}_{0}}{{W*W} + {H*H}}W*W} - {\frac{{vx}_{3} - {vx}_{0}}{{W*W} + {H*H}}H*W} + {vy}_{0}}}\end{matrix} \right. & (16)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx}_{2} = {{\frac{{vx}_{3} - {vx}_{0}}{{W*W} + {H*H}}H*H} - {\frac{{vy}_{3} - {vy}_{0}}{{W*W} + {H*H}}H*W} + {vx}_{0}}} \\{{vy}_{2} = {{\frac{{vy}_{3} - {vy}_{0}}{{W*W} + {H*H}}W*H} + {\frac{{vx}_{3} - {vx}_{0}}{{W*W} + {H*H}}H*H} + {vy}_{0}}}\end{matrix} \right. & (17)\end{matrix}$

{CP2, CP4} may be converted into {CP1, CP2} according to Formula (18),and {CP2, CP4} may be converted into {CP1, CP2, CP3} according toFormulas (18) and (19):

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{0} = {{{- \frac{{vy}_{3} - {vy}_{1}}{H}}W} + {vx}_{1}}} \\{{vy}_{0} = {{{+ \frac{{vx}_{3} - {vx}_{1}}{H}}W} + {vy}_{1}}}\end{matrix} \right. & (18)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx}_{2} = {{{- \frac{{vy}_{3} - {vy}_{1}}{H}}W} + {vx}_{3}}} \\{{vy}_{2} = {{{+ \frac{{vx}_{3} - {vx}_{1}}{H}}W} + {vy}_{3}}}\end{matrix} \right. & (19)\end{matrix}$

{CP3, CP4} may be converted into {CP1, CP2} according to Formula (20),and {CP3, CP4} may be converted into {CP1, CP2, CP3} according toFormulas (20) and (21):

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{0} = {{{+ \frac{{vy}_{3} - {vy}_{2}}{W}}H} + {vx}_{2}}} \\{{vy}_{0} = {{{- \frac{{vx}_{3} - {vx}_{2}}{W}}H} + {vy}_{2}}}\end{matrix} \right. & (20)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx}_{1} = {{{+ \frac{{vy}_{3} - {vy}_{2}}{W}}H} + {vx}_{3}}} \\{{vy}_{1} = {{{- \frac{{vx}_{3} - {vx}_{2}}{W}}H} + {vy}_{3}}}\end{matrix} \right. & (21)\end{matrix}$

For example, a 6-parameter affine motion model obtained based on acombination {CP1, CP2, CP4}, {CP2, CP3, CP4}, or {CP1, CP3, CP4} isconverted and represented by using {CP1, CP2, CP3}. A conversion methodcomprises: substituting motion vectors and coordinate information ofcontrol points into Formula (4) to obtain model parameters; and thensubstituting coordinate information of {CP1, CP2, CP3} into Formula (5)to obtain motion vectors of {CP1, CP2, CP3}.

More directly, conversion may be performed according to Formulas (22) to(24), where W represents the width of the current block, and Hrepresents the height of the current block. In Formulas (13) to (21),(vx₀, vy₀) represents a motion vector of CP1, (vx₁, vy₁) represents amotion vector of CP2, (vx₂, vy₂) represents a motion vector of CP3, and(vx₃, vy₃) represents a motion vector of CP4.

{CP1, CP2, CP4} may be converted into {CP1, CP2, CP3} according toFormula (22):

$\begin{matrix}\left\{ \begin{matrix}{{vx}_{2} = {{vx}_{3} + {vx}_{0} - {vx}_{1}}} \\{{vy}_{2} = {{vy}_{3} + {vy}_{0} - {vy}_{1}}}\end{matrix} \right. & (22)\end{matrix}$

{CP2, CP3, CP4} may be converted into {CP1, CP2, CP3} according toFormula (23):

$\begin{matrix}\left\{ \begin{matrix}{{vx_{0}} = {{vx_{1}} + {vx_{2}} - {vx_{3}}}} \\{{vy_{0}} = {{vy_{1}} + {vy_{2}} - {vy_{3}}}}\end{matrix} \right. & (23)\end{matrix}$

{CP1, CP3, CP4} may be converted into {CP1, CP2, CP3} according toFormula (24):

$\begin{matrix}\left\{ \begin{matrix}{{vx_{1}} = {{vx_{3}} + {vx_{0}} - {vx_{2}}}} \\{{vy_{1}} = {{vy_{3}} + {vy_{0}} - {vy_{2}}}}\end{matrix} \right. & (24)\end{matrix}$

6. Advanced Temporal Motion Vector Prediction (Advanced Temporal MotionVector Prediction, ATMVP):

In inter prediction in HEVC, motion compensation is performed on allsamples of a current block based on same motion information to obtainpredictors of the samples of the to-be-processed block. However, not allthe samples of the to-be-processed block have same motion features.Using the same motion information to predict all the samples of theto-be-processed block may reduce motion compensation accuracy andincrease residual information.

To resolve the foregoing problems, an advanced temporal motion vectorprediction (ATMVP) technology is provided in an existing solution.

A process of performing prediction by using the ATMVP technology mainlycomprises the following operations, as shown in FIG. 6E:

(1) Determine an offset vector of the to-be-processed block.

(2) Determine, in a corresponding target picture based on a location ofa to-be-processed subblock of the to-be-processed block and the offsetvector, a subblock corresponding to the to-be-processed subblock, wherethe target picture is one of encoded pictures.

(3) Determine a motion vector of the to-be-processed subblock based on amotion vector of the corresponding subblock.

For example, the motion vector of the current to-be-processed subblockmay be determined by using a scaling method. For example, the scalingmethod is implemented according to Formula (25):

$\begin{matrix}{{MV}_{c} = {\frac{{CPoc} - {DPoc}}{{DPoc} - {SPoc}} \times {MV}_{g}}} & (25)\end{matrix}$

where CPoc represents a POC number of a frame in which theto-be-processed block is located, DPoc represents a POC number of aframe in which the corresponding subblock is located, SrcPoc representsa POC number of a reference frame of the corresponding subblock, MV_(c)represents a motion vector obtained through scaling, and MV_(g)represents the motion vector of the corresponding subblock.

(4) Perform motion compensation prediction on the to-be-processedsubblock based on the motion vector of the to-be-processed subblock, toobtain a prediction sample value of the to-be-processed subblock.

7. Planar Motion Vector Prediction (PLANAR):

In planar motion vector prediction, motion information on a top spatialneighboring location, a left spatial neighboring location, a rightlocation, and a bottom location of each to-be-processed subblock of theto-be-processed block is obtained, and an average of the motioninformation is calculated, and converted into motion information of thecurrent to-be-processed subblock.

For a to-be-processed subblock whose coordinates are (x, y), a motionvector P(x, y) of the to-be-processed subblock is calculated based on ahorizontal interpolation motion vector P_(h)(x, y) and a horizontalinterpolation motion vector P_(v)(x, y) according to Formula (26):

P(x,y)=(H×P _(h)(x,y)+W×P _(v)(x,y)+H×W)/(2×H×W)  (26)

where H represents the height of the to-be-processed block, and Wrepresents the width of the to-be-processed block.

The horizontal interpolation motion vector P_(h)(x, y) and thehorizontal interpolation motion vector P_(v)(x, y) may be calculatedbased on motion vectors of subblocks on the left, the right, the top,and the bottom of the current to-be-processed subblock according toFormulas (27) and (28):

P _(h)(x,y)=(W−1−x)−L(−1,y)+(x+1)×R(w,y)  (27)

P _(v)(x,y)=(H−1−y)×A(x,−1)+(y+1)×B(x,H)  (28)

where L(−1, y) represents a motion vector of a subblock on the left ofthe to-be-processed subblock, R(w, y) represents a motion vector of asubblock on the right of the to-be-processed subblock, A(x, −1)represents a motion vector of a subblock on the top of theto-be-processed subblock, and B(x, H) represents a motion vector of asubblock on the bottom of the to-be-processed subblock.

A motion vector L of a left block and a motion vector A of a top blockare obtained based on a spatial neighboring block of a current codingblock. Motion vectors L(−1, y) and A(x, −1) of coding blocks at presetlocations (−1, y) and (x, −1) are obtained based on the coordinates (x,y) of the to-be-processed subblock.

As shown in FIG. 7, a motion vector R(w, y) of a right block and amotion vector B (x, H) of a bottom block may be extracted by using thefollowing method:

(1) Extract temporal motion information BR on a bottom-right spatialneighboring location of the to-be-processed block.

(2) Obtain the motion vector R(w, y) of the right block by performingweighting calculation based on extracted motion vector AR on a top-rightspatial neighboring location and the extracted temporal motioninformation BR on the bottom-right spatial neighboring location, asshown in Formula (29):

R(w,y)=((H−y−1)*AR+(y+1)*BR)/H  (29)

(3) Obtain the motion vector B(x, H) of the bottom block by performingweighting calculation based on extracted motion vector BL on abottom-left spatial neighboring location and the extracted temporalmotion information BR on the bottom-right spatial neighboring location,as shown in Formula (30):

B(x,H)=((W−x−1)*BL+(x+1)*BR)/H  (30)

It should be noted that the motion vector used in the calculation is amotion vector obtained through scaling after the motion vector is scaledto point to the first reference frame in a specific reference framequeue.

8. Affine Motion Model Based Advanced Motion Vector Prediction Mode(Affine AMVP Mode):

(1) Construct a candidate motion vector list.

A candidate motion vector list corresponding to the affine motion modelbased AMVP mode is constructed by using the inherited control pointmotion vector prediction method and/or the constructed control pointmotion vector prediction method. In the embodiments of this application,the candidate motion vector list corresponding to the affine motionmodel based AMVP mode may be referred to as a control point motionvector predictor candidate list (control point motion vectors predictorcandidate list). A motion vector predictor of each control pointcomprises motion vectors of two control points (for a 4-parameter affinemotion model) or motion vectors of three control points (for a6-parameter affine motion model).

Optionally, the control point motion vector predictor candidate list ispruned and sorted according to a particular rule, and may be truncatedor padded to a particular quantity.

(2) Determine an optimal control point motion vector predictor.

On an encoder side, a motion vector of each subblock of a current codingblock is obtained based on each control point motion vector predictor inthe control point motion vector predictor candidate list according toFormula (3) or (5). The obtained motion vector is used to obtain asample value on a corresponding location in a reference frame to whichthe motion vector of the subblock points. The obtained sample value isused as a predictor to perform motion compensation using an affinemotion model. An average difference between an original value and aprediction value of each sample of the current coding block iscalculated. A control point motion vector predictor corresponding to aminimum average difference is selected as the optimal control pointmotion vector predictor, and used as motion vector predictors of two orthree control points of the current coding block. An index numberrepresenting a location of the control point motion vector predictor inthe control point motion vector predictor candidate list is encoded in abitstream and sent to a decoder.

On the decoder side, the index number is parsed, and the control pointmotion vector predictor (CPMVP) is determined from the control pointmotion vector predictor candidate list based on the index number.

(3) Determine a control point motion vector

On the encoder side, a control point motion vector predictor is used asa search start point for motion search within a specific search range,to obtain a control point motion vector (CPMV). A difference between thecontrol point motion vector and the control point motion vectorpredictor (control point motion vectors differences, CPMVD) istransferred to the decoder side.

On the decoder side, the control point motion vector difference isparsed and added to the control point motion vector predictor, to obtainthe control point motion vector.

9. Subblock Merge Mode (Sub-Block Based Merging Mode):

A subblock-based merging candidate list (sub-block based mergingcandidate list) is constructed by using at least one of the advancedtemporal motion vector prediction, the inherited control point motionvector prediction method, the constructed control point motion vectorprediction method, and the planar prediction method.

Optionally, the subblock-based merging candidate list is pruned andsorted according to a particular rule, and may be truncated or padded toa particular quantity.

On an encoder side, if the advanced temporal motion vector prediction isused, a motion vector of each subblock (a sample or a N₁×N₂ sample blockobtained by using a particular partitioning method) is obtained by usingthe method described in the foregoing descriptions in “7. Planar motionvector prediction”. If the planar prediction method is used, a motionvector of each subblock is obtained by using the method described in theforegoing descriptions in “8. Affine motion model based advanced motionvector prediction mode”.

If the inherited control point motion vector prediction method or theconstructed control point motion vector prediction method is used, amotion vector of each subblock (a sample or a N₁×N₂ sample blockobtained by using a particular partitioning method) of a current blockis obtained according to Formula (3) or (5). After the motion vector ofeach subblock is obtained, a sample value on a location in a referenceframe to which the motion vector of the subblock points is furtherobtained, and the sample value is used as a predictor of the subblockfor affine motion compensation. An average difference between anoriginal value and a predictor of each sample of a current coding blockis calculated. A control point motion vector corresponding to a minimumaverage difference is selected as motion vectors of two or three controlpoints of the current coding block. An index number representing alocation of the control point motion vector in the candidate list isencoded in a bitstream and sent to a decoder.

On the decoder side, the index number is parsed, and the control pointmotion vector (CPMV) is determined from the control point motion vectormerging candidate list based on the index number.

In addition, it should be noted that in this application, “at least one”means one or more, and “a plurality of” means two or more than two. Theterm “and/or” describes an association relationship for describingassociated objects and represents that three relationships may exist.For example, A and/or B may represent the following cases: Only Aexists, both A and B exist, and only B exists, where A and B may besingular or plural. The character “/” generally represents an “or”relationship between the associated objects. “At least one item (piece)of the following” or a similar expression thereof means any combinationof these items, including a singular item (piece) or any combination ofplural items (pieces). For example, at least one item (piece) of a, b,or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and cmay be singular or plural.

In an embodiment this application, when an inter prediction mode is usedto decode a current block, a syntax element may be used to signal theinter prediction mode.

For some currently used syntax structures of the inter prediction modeused to parse the current block, refer to Table 1. It should be notedthat a syntax element in a syntax structure may alternatively berepresented by other indicators. This is not specifically limited inthis application.

TABLE 1 coding unit(x0, y0, cbWidth, cbHeight, treeType) { Descriptor if(slice type != I) {   cu_skip_flag[x0][y0] ae(v)  if(cu_skip_flag[x0][y0] == 0)    pred_mode_flag[x0][y0] ae(v)  } if(CuPredMode[x0][y0] == MODE_INTRA){   if(treeType == SINGLE TREE | |treeType == DUAL_TREE_LUMA) {    intra_luma_mpm_flag[x0][y0] ae(v)   if(intra_luma_mpm_flag[x0][y0])     intra_luma_mpm_idx[x0][y0] ae(v)   else     intra_luma_mpm_remainder[x0][y0] ae(v)   }   if(treeType ==SINGLE_TREE | | treeType == DUAL_TREE_CHROMA)   intra_chroma_pred_mode[x0][y0] ae(v)  } else { /* MODE_INTER (intermode) */   if(cu_skip_flag[x0][y0]) {    if(MaxNumSubblockMergeCand > 0&& cbWidth >= 8 && cbHeight >= 8)     merge_subblock_flag[x0][y0] ae(v)   if(merge_subblock_flag[x0][y0] == 0 && MaxNumMergeCand > 1)    merge_idx[x0][y0] ae(v)    if(merge_subblock_flag[x0][y0] == 1 &&MaxNumSubblockMergeCand > 1)     merge_subblock_idx[x0][y0] ae(v)   }else {    merge_flag[x0][y0] ae(v)    if(merge_flag[x0][y0]) {    if((sps_affine_enabled_flag | | sps_sbtmvp_enabled _flag) &&cbWidth >= 8 && cbHeight >= 8)      merge_subblock_flag[x0][y0] ae(v)    if(merge_subblock_flag[x0][y0] == 0 && MaxNumMergeCand > 1)     merge_idx[x0][y0] ae(v)    if(merge_subblock_flag[x0][y0] == 1)    merge_subblock_idx[x0][y0] ae(v)    } else {     if(slice_type == B)     inter_pred_idc[x0][y0] ae(v)     if(sps_affine_enabled_flag &&cbWidth >= 16 && cbHeight >= 16) {      inter_affine_flag[x0][y0] ae(v)     if(sps_affine_type_flag && inter_affine_flag[x0][y0])      cu_affine_type _flag[x0][y0] ae(v)      }     if(inter_pred_idc[x0][y0] != PRED_L1) {     if(num_ref_idx_l0_active_ minus1 > 0)       ref_idx_l0[x0][y0]ae(v)      mvd_coding(x0, y0, 0, 0)      if(MotionModelIdc[x0][y0] > 0)      mvd_coding(x0, y0, 0, 1)      if(MotionModelIdc[x0][y0] > 1)      mvd_coding(x0, y0, 0, 2)      mvp_l0_flag[x0][y0] ae(v)     } else{      MvdL0[x0][y0][0] = 0      MvdL0[x0][y0][1] = 0     }    if(inter_pred_idc[x0][y0] != PRED_L0) {     if(num_ref_idx_l1_active_minus1 > 0)       ref_idx_l1[x0][y0] ae(v)     if(mvd_l1_zero_flag && inter_pred_idc[x0][y0] == PRED_BI) {      MvdL1[x0][y0][0] = 0       MvdL1[x0][y0][1] = 0      MvdCpL1[x0][y0][0][0] = 0       MvdCpL1[x0][y0][0][1] = 0      MvdCpL1[x0][y0][1][0] = 0       MvdCpL1[x0][y0][1][1] = 0      MvdCpL1[x0][y0][2][0] = 0       MvdCpL1[x0][y0][2][1] = 0     }else {       mvd_coding(x0, y0, 1, 0)      if(MotionModelIdc[x0][y0] >0)       mvd_coding(x0, y0, 1, 1)      if(MotionModelIdc[x0][y0] > 1)      mvd_coding(x0, y0, 1, 2)      mvp_l1_flag[x0][y0] ae(v)     } else{      MvdL1[x0][y0][0] = 0      MvdL1[x0][y0][1] = 0     }    if(sps_amvr_enabled_flag && inter_affine_flag == 0 &&     (MvdL0[x0][y0][0] != 0 | | MvdL0[x0][y0][1] != 0 | |      MvdL1[x0][y0][0] != 0 | | MvdL1[x0][y0][1] != 0))     amvr_mode[x0][y0] ae(v)    }   }  }  if(CuPredMode[x0][y0] !=MODE_INTRA && cu_skip_flag[x0][y0] == 0)   cu_cbf ae(v)  if(cu_cbf) {  transform_tree(x0, y0, cbWidth, cbHeight, treeType) }

The variable treeType specifies a coding tree type used to encode acurrent block.

The variable slice_type is used to specify a type of a slice in which acurrent block is located, for example, a P type, a B type, or an I type.

The syntax element cu_skip_flag[x0][y0] may be used to specify whether acurrent block has a residual. For example, when cu_skip_flag[x0][y0]=1,it indicates that the current block has the residual; or whencu_skip_flag[x0][y0]=0, it indicates that the current block has noresidual.

It should be noted that a skip mode is a special merge mode. After amerge mode is used to find a motion vector, if an encoder determines, byusing a method, that the current block is basically the same as areference block, residual data does not need to be transmitted, and onlyan index of the motion vector and a cu_skip_flag need to be transmitted.Therefore, if cu_skip_flag[x0][y0]=0, it indicates that the currentblock has no residual and residual data does not need to be parsed.

The syntax element pred_mode_flag[x0][y0] is used to specify whether aprediction mode for a current block is an inter prediction or intraprediction mode.

The variable CuPredMode[x0] [y0] is determined based onpre_mode_flag[x0][y0]. MODE_INTRA specifies an intra prediction mode.

The syntax element merge_flag[x0][y0] may be used to specify whether amerge (merge) mode is used for a current block. For example, whenmerge_flag[x0][y0]=1, it indicates that the merge mode is used for thecurrent block; or when merge_flag[x0][y0]=0, it indicates that the mergemode is not used for the current block, where x0 and y0 representcoordinates of the current block in a video picture.

cbWidth specifies the width of the current block, and cbHeight specifiesthe height of the current block.

The syntax element sps_affine_enabled_flag may be used to specifywhether an affine motion model may be used to inter predict on a pictureblock comprised in a video picture. For example, whensps_affine_enabled_flag=0, it indicates that the affine motion modelcannot be used to inter predict the picture block comprised in the videopicture; or when sps_affine_enabled_flag=1, it indicates that the affinemotion model can be used to inter predict the picture block comprised inthe video picture.

The syntax element merge_subblock_flag[x0][y0] may be used to specifywhether a subblock-based merge mode is used for a current block. A type(slice_type) of a slice in which the current block is located is a Ptype or a B type. For example, when merge_subblock_flag[x0][y0]=1, itindicates that the subblock-based merge mode is used for the currentblock; or when merge_subblock_flag[x0][y0]=0, it indicates that thesubblock-based merge mode is not used for the current block, but atranslational motion model based merge mode can be used.

The syntax element merge_idx[x0][y0] may be used to specify an index ofa merging candidate list.

The syntax element merge_subblock_idx[x0][y0] may be used to specify anindex of a subblock-based merging candidate list.

sps_sbtmvp_enabled_flag may be used to specify whether an ATMVP mode canbe used to inter predict a picture block comprised in a video picture.For example, when sps_sbtmvp_enabled_flag=1, it indicates that the ATMVPmode can be used to inter predict the picture block comprised in thevideo picture; or when sps_sbtmvp_enabled_flag=0, it indicates that theATMVP mode cannot be used to inter predict the picture block comprisedin the video picture.

The syntax element inter_affine_flag[x0][y0] may be used to specifywhether an affine motion model based AMVP mode is used for a currentblock when a slice in which the current block is located is a P-typeslice or a B-type slice. For example, when inter_affine_flag[x0][y0]=0,it indicates that the affine motion model based AMVP mode is used forthe current block; or when inter_affine_flag[x0][y0]=1, it indicatesthat the affine motion model based AMVP mode is not used for the currentblock, but a translational motion model based AMVP mode can be used.

The syntax element cu_affine_type_flag[x0][y0] may be used to specifywhether a 6-parameter affine motion model is used to perform motioncompensation for a current block when a slice in which the current blockis located is a P-type slice or a B-type slice. Whencu_affine_type_flag[x0][y0]=0, it indicates that the 6-parameter affinemotion model is not used to perform motion compensation for the currentblock, but only a 4-parameter affine motion model may be used to performmotion compensation; or when cu_affine_type_flag[x0][y0]=1, it indicatesthat the 6-parameter affine motion model is used to perform motioncompensation for the current block.

As shown in Table 2, when MotionModelIdc[x0][y0]=1, it indicates that a4-parameter affine motion model is used; when MotionModelIdc[x0][y0]=2,it indicates that a 6-parameter affine motion model is used; or whenMotionModelIdc[x0][y0]=0, it indicates that a translational motion modelis used.

TABLE 2 Motion model for motion compensation (motion modelMotionModelIdc[x0][y0] for motion compensation) 0 Translational motion(translational motion) 1 4-parameter affine motion (4-parameter affinemotion) 2 6-parameter affine motion (6-parameter affine motion)

The variable MaxNumMergeCand is used to specify a maximum length of amerging candidate motion vector list, the variableMaxNumSubblockMergeCand is used to specify a maximum length of asubblock-based merging candidate motion vector list, inter_pred_idc[x0][y0] is used to specify a prediction direction, PRED_L0 specifiesforward prediction, num_ref_idx_l0_active_minus1 specifies the number ofreference frames in a forward reference frame list, ref_idx_l0[x0][y0]specifies an index value for a forward reference frame of a currentblock, mvd_coding(x0, y0, 0, 0) specifies the first motion vectordifference, mvp_l0_flag[x0][y0] specifies an index value for a forwardMVP candidate list, PRED_L1 is used to indicate backward prediction,num_ref_idx_l1_active_minus1 indicates the number of reference frames ina backward reference frame list, ref_idx_l1[x0][y0] specifies an indexvalue for a backward reference frame of the current block, andmvp_l1_flag[x0][y0] specifies an index value for a backward MVPcandidate list.

In Table 1, ae(v) represents a syntax element encoded by using acontext-based adaptive binary arithmetic coding (CABAC).

The following describes an inter prediction process in detail accordingto some embodiments, as shown in FIG. 8A.

Operation 801: Parse a bitstream based on a syntax structure shown inTable 1, and determine an inter prediction mode for a current block.

If it is determined that the inter prediction mode for the current blockis an affine motion model based AMVP mode, operation 802 a is performed.

The syntax elements sps_affine_enabled_flag=1, merge_flag=0, andinter_affine_flag=1 indicate that the inter prediction mode for thecurrent block is the affine motion model based AMVP mode.

If it is determined that the inter prediction mode for the current blockis a subblock merge (merge) mode, operation 802 b is performed.

The syntax element sps_affine_enabled_flag=1 or the syntax elementsps_sbtmvp_enabled_flag=1, and the syntax elements merge_flag=1 andmerge_subblock_flag=1 indicate that the inter prediction mode for thecurrent block is the subblock merge mode.

Operation 802 a: Construct a candidate motion vector list correspondingto the affine motion model based AMVP mode. Perform operation 803 a.

A candidate control point motion vector of the current block is derivedby using an inherited control point motion vector prediction methodand/or a constructed control point motion vector prediction method, andis added to the candidate motion vector list.

The candidate motion vector list may comprise a 2-tuple list (a4-parameter affine motion model is used for the current coding block) ora 3-tuple list. The 2-tuple list comprises one or more 2-tuples used toconstruct a 4-parameter affine motion model. The 3-tuple list comprisesone or more 3-tuples used to construct a 6-parameter affine motionmodel.

Optionally, the candidate motion vector 2-tuple/3-tuple list is prunedand sorted according to a particular rule, and may be truncated orpadded to a particular quantity.

A1: A process of constructing the candidate motion vector list by usingthe inherited control point motion vector prediction method isdescribed.

FIG. 4 is used as an example. In the example in FIG. 4, blocks atneighboring locations around the current block are traversed in an orderof A1->B1->B0->A0->B2, to find an affine coded block including a blockat a neighboring location of the current block, and to obtain controlpoint motion information of the affine coded block. Further, the controlpoint motion information of the affine coded block is used to constructa motion model, to derive candidate control point motion information ofthe current block. For details, refer to the foregoing descriptions in“3. Inherited control point motion vector prediction method”. Detailsare not described herein again.

For example, an affine motion model used for the current block is a4-parameter affine motion model (that is, MotionModelIdc=1). If a4-parameter affine motion model is used for a neighboring affinedecoding block, motion vectors of two control points of the affinedecoding block are obtained: a motion vector (vx₄, vy₄) of the top-leftcontrol point (x₄, y₄) and a motion vector (vx₅, vy₅) of the top-rightcontrol point (x₅, y₅). The affine decoding block is an affine codedblock for which prediction is performed by using an affine motion modelduring encoding.

Motion vectors of two control points, namely, the top-left and top-rightcontrol points of the current block are derived respectively accordingto Formulas (6) and (7) corresponding to the 4-parameter affine motionmodel, by using the 4-parameter affine motion model including the twocontrol points of the neighboring affine decoding block.

If a 6-parameter affine motion model is used for the neighboring affinedecoding block, motion vectors of three control points of theneighboring affine decoding block are obtained, for example, a motionvector (vx₄, vy₄) of the top-left control point (x₄, y₄), a motionvector (vx₅, vy₅) of the top-right control point (x₅, y₅), and a motionvector (vx₆, vy₆) of the bottom-left vertex (x₆, y₆) in FIG. 4.

Motion vectors of two control points, namely, the top-left and top-rightcontrol points of the current block are derived respectively accordingto Formulas (8) and (9) corresponding to the 6-parameter affine motionmodel, by using the 6-parameter affine motion model including the threecontrol points of the neighboring affine decoding block.

For example, an affine motion model used for a current decoding block isa 6-parameter affine motion model (that is, MotionModelIdc=2). If anaffine motion model used for a neighboring affine decoding block is a6-parameter affine motion model, motion vectors of three control pointsof the neighboring affine decoding block are obtained, for example, amotion vector (vx₄, vy₄) of the top-left control point (x₄, y₄), amotion vector (vx₅, vy₅) of the top-right control point (x₅, y₅), and amotion vector (vx₆, vy₆) of the bottom-left vertex (x₆, y₆) in FIG. 4.

Motion vectors of three control points, namely, the top-left, top-right,and bottom-left control points of the current block are derivedrespectively according to Formulas (8), (9), and (10) corresponding tothe 6-parameter affine motion model, by using the 6-parameter affinemotion model including the three control points of the neighboringaffine decoding block.

If an affine motion model used for the neighboring affine decoding blockis a 4-parameter affine motion model, motion vectors of two controlpoints of the neighboring affine decoding block are obtained: a motionvector (vx₄, vy₄) of the top-left control point (x₄, y₄) and a motionvector (vx₅, vy₅) of the top-right control point (x₅, y₅).

Motion vectors of three control points, namely, the top-left, top-right,and bottom-left control points of the current block are derivedrespectively according to Formulas (6) and (7) corresponding to the4-parameter affine motion model, by using the 4-parameter affine motionmodel including the two control points of the neighboring affinedecoding block.

It should be noted that other motion models, candidate locations, andsearch orders may also be used in embodiments of this application.Details are not described herein. It should be noted that methods forrepresenting motion models of neighboring and current coding blocksbased on other control points may also be used in embodiments of thisapplication. Details are not described herein.

A2: A process of constructing the candidate motion vector list by usingthe constructed control point motion vector prediction method isdescribed.

For example, an affine motion model used for a current decoding block isa 4-parameter affine motion model (that is, MotionModelIdc=1), andmotion vectors of the top-left vertex and the top-right vertex of thecurrent coding block are determined based on motion information of aneighboring encoded block of the current coding block. Specifically, thecandidate motion vector list may be constructed by using the constructedcontrol point motion vector prediction method 1 or the constructedcontrol point motion vector prediction method 2. For the specificmethod, refer to the foregoing descriptions in “4. Constructed controlpoint motion vector prediction method 1” and “5. Constructed controlpoint motion vector prediction method 2”. Details are not describedherein again.

For example, an affine motion model used for a current decoding block isa 6-parameter affine motion model (that is, MotionModelIdc=2), andmotion vectors of the top-left vertex, the top-right vertex, and thebottom-left vertex of the current coding block are determined based onmotion information of a neighboring encoded block of the current codingblock. Specifically, the candidate motion vector list may be constructedby using the constructed control point motion vector prediction method 1or the constructed control point motion vector prediction method 2. Forthe specific method, refer to the foregoing descriptions in “4.Constructed control point motion vector prediction method 1” and “5.Constructed control point motion vector prediction method 2”. Detailsare not described herein again.

It should be noted that other combinations of control point motioninformation may also be used in embodiments of this application. Detailsare not described herein.

Operation 803 a: Parse the bitstream and determine an optimal controlpoint motion vector predictor. Perform operation 804 a.

B1: If the affine motion model used for the current decoding block isthe 4-parameter affine motion model (MotionModelIdc=1), an index numberis parsed, and an optimal motion vector predictor for the two controlpoints are determined from the candidate motion vector list based on theindex number.

For example, the index number is mvp_l0_flag or mvp_l1_flag.

B2: If the affine motion model used for the current decoding block isthe 6-parameter affine motion model (MotionModelIdc=2), an index numberis parsed, and an optimal motion vector predictor for the three controlpoints are determined from the candidate motion vector list based on theindex number.

Operation 804 a: Parse the bitstream and determine a control pointmotion vector.

C1: If the affine motion model used for the current decoding block isthe 4-parameter affine motion model (MotionModelIdc=1), motion vectordifferences of the two control points of the current block are obtainedby decoding the bitstream, and motion vectors of the two control pointsare then obtained based on the motion vector differences and the motionvector predictors of the control points. Using forward prediction as anexample, the motion vector differences of the two control points aremvd_coding(x0, y0, 0, 0) and mvd_coding(x0, y0, 0, 1), respectively.

For example, motion vector differences of the top-left control point andthe top-right control point are obtained by decoding the bitstream, andare added to respective motion vector predictors, to obtain motionvectors of the top-left control point and the top-right control point ofthe current block.

C2: If the affine motion model used for the current decoding block isthe 6-parameter affine motion model (MotionModelIdc=2), motion vectordifferences of the three control points of the current block areobtained by decoding the bitstream, and motion vectors of the threecontrol points are then obtained based on the motion vector differencesand the motion vector predictors of the control points. Using forwardprediction as an example, motion vector differences of three controlpoints are mvd_coding(x0, y0, 0, 0), mvd_coding(x0, y0, 0, 1), andmvd_coding(x0, y0, 0, 2), respectively.

For example, motion vector differences of the top-left control point,the top-right control point, and the bottom-left control point areobtained by decoding the bitstream, and are added to respective motionvector predictors, to obtain motion vectors of the top-left controlpoint, the top-right control point, and the bottom-left control point ofthe current block.

Operation 805 a: Obtain a motion vector of each subblock of the currentblock based on control point motion information and the affine motionmodel used for the current decoding block.

One subblock in the current affine decoding block may be equivalent toone motion compensation unit, and the width and the height of thesubblock are less than the width and the height of the current block.Motion information of a sample at a preset location in a motioncompensation unit may be used to represent motion information of allsamples of the motion compensation unit. Assuming that the size of themotion compensation unit is M×N, the sample at the preset location maybe a center sample (M/2, N/2), the top-left sample (0, 0), the top-rightsample (M−1, 0), or a sample at another location in the motioncompensation unit. The center sample of the motion compensation unit isused as an example for description below, as shown in FIG. 8C. In FIG.8C, Vo represents a motion vector of the top-left control point, and Virepresents a motion vector of the top-right control point. Each smallbox represents one motion compensation unit.

Coordinates of a center sample of a motion compensation unit relative tothe top-left sample of the current affine decoding block are calculatedaccording to Formula (31). In Formula (31), i represents the i^(th)motion compensation unit in the horizontal direction (from left toright), j represents the i^(th) motion compensation unit in the verticaldirection (from top to bottom), and (x_((i,j)), y_((i,j))) representscoordinates of the center sample of the (i, j)^(th) motion compensationunit relative to the top-left control point sample of the current affinedecoding block.

If the affine motion model used for the current affine decoding block isthe 6-parameter affine motion model, (x_((i,j)), y_((i,j))) issubstituted into Formula (32) corresponding to the 6-parameter affinemotion model to obtain a motion vector (vx_((i,j)), vy_((i,j))) of acenter sample of each motion compensation unit, and the motion vector isused as motion vectors of all samples of the motion compensation unit.

If the affine motion model used for the current affine decoding block isthe 4-parameter affine motion model, (x_((i,j)), y_((i,j))) issubstituted into Formula (33) corresponding to the 4-parameter affinemotion model to obtain a motion vector (vx_((i,j)), vy_((i,j))) of acenter sample of each motion compensation unit, and the motion vector isused as motion vectors of all samples of the motion compensation unit.

$\begin{matrix}\left\{ \begin{matrix}{{x_{({i,j})} = {{M \times i} + \frac{M}{2}}},\ {i = 0},{1\ldots}} \\{{y_{({i,j})} = {{N \times j} + \frac{N}{2}}},{j = 0},{1\ldots}}\end{matrix} \right. & (31)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx} = {{\frac{{vx}_{1} - {vx}_{0}}{W}x} + {\frac{{vx}_{2} - {vy}_{0}}{H}y} + {vx}_{0}}} \\{{vy} = {{\frac{{vy}_{1} - {vy}_{0}}{W}x} + {\frac{{vy}_{2} - {vx}_{0}}{H}y} + {vy}_{0}}}\end{matrix} \right. & (32)\end{matrix}$ $\begin{matrix}\left\{ \begin{matrix}{{vx} = {{\frac{{vx}_{1} - {vx}_{0}}{W}x} - {\frac{{vy}_{1} - {vy}_{0}}{W}y} + {vx}_{0}}} \\{{vy} = {{\frac{{vy}_{1} - {vy}_{0}}{W}x} + {\frac{{vx}_{1} - {vx}_{0}}{W}y} + {vy}_{0}}}\end{matrix} \right. & (33)\end{matrix}$

Operation 806 a: Perform motion compensation for each subblock based onthe determined motion vector of the subblock, to obtain a predictionsample value of the subblock.

Operation 802 b: Construct a motion information candidate listcorresponding to a subblock merge (merge) mode.

Specifically, the motion information candidate list corresponding to thesubblock merge mode (sub-block based merging candidate list) may beconstructed by using one or more of advanced temporal motion vectorprediction, an inherited control point motion vector prediction method,a constructed control point motion vector prediction method, or a planarmethod. In this embodiment of this application, the motion informationcandidate list corresponding to the subblock merge mode may be referredto as a subblock-based merging candidate list for short.

Optionally, the motion information candidate list is pruned and sortedaccording to a particular rule, and may be truncated or padded to aparticular quantity.

D0: If sps_sbtmvp_enabled_flag=1, a candidate motion vector is obtainedby using the ATMVP, and added to the subblock-based merging candidatelist. For details, refer to the foregoing descriptions in “6. Advancedtemporal motion vector prediction”.

D1: If sps_affine_enabled_flag=1, candidate control point motioninformation of the current block is derived by using the inheritedcontrol point motion vector prediction method, and is added to thesubblock-based merging candidate list. For details, refer to theforegoing descriptions in “3. Inherited control point motion vectorprediction method”.

For example, in FIG. 6C, blocks at neighboring locations around thecurrent block are traversed in an order of A1->B1->B0->A0->B2, to findan affine coded block including a block at a neighboring location of thecurrent block, and to obtain control point motion information of theaffine coded block. Further, a motion model corresponding to the affinecoded block is used to derive candidate control point motion informationof the current block.

If the subblock-based merging candidate list is empty, the candidatecontrol point motion information is added to the subblock-based mergingcandidate list. Otherwise, motion information in the subblock-basedmerging candidate list is traversed sequentially to check whether motioninformation that is the same as the candidate control point motioninformation exists in the subblock-based merging candidate list. If nomotion information that is the same as the candidate control pointmotion information exists in the subblock-based merging candidate list,the candidate control point motion information is added to thesubblock-based merging candidate list.

Determining whether two pieces of candidate motion information are thesame is to be implemented by determining whether forward and backwardreference frames of the two pieces of candidate motion information andhorizontal and vertical components of each forward motion vector andbackward motion vector are the same. The two pieces of candidate motioninformation are considered as being different only when all theseelements are different.

If a quantity of pieces of motion information in the subblock-basedmerging candidate list reaches a maximum list lengthMaxNumSubblockMergeCand (MaxNumSubblockMergeCand is a positive integersuch as 1, 2, 3, 4, or 5, where 5 is used as an example in the followingdescriptions, and details are not described herein), construction of thecandidate list is completed. Otherwise, a block at a next neighboringlocation is traversed.

D2: If sps_affine_enabled_flag=1, candidate control point motioninformation of the current block is derived by using the constructedcontrol point motion vector prediction method, and is added to thesubblock-based merging candidate list according to some embodiments, asshown in FIG. 8B.

Operation 801 c: Obtain motion information of control points of thecurrent block. For example, for details, refer to operation 601 in theforegoing descriptions in “5. Constructed control point motion vectorprediction method 2”. The details are not described herein again.

Operation 802 c: Combine the motion information of the control points toobtain constructed control point motion information. For details, referto operation 602 in FIG. 6D. The details are not described herein again.

Operation 803 c: Add the constructed control point motion information tothe subblock-based merging candidate list.

If the length of the subblock-based merging candidate list is less thana maximum list length MaxNumSubblockMergeCand, the combinations of themotion information of the control points are traversed in a presetorder, and a resulting valid combination is used as the candidatecontrol point motion information. In this case, if the subblock-basedmerging candidate list is empty, the candidate control point motioninformation is added to the subblock-based merging candidate list.Otherwise, motion information in the candidate motion vector list istraversed sequentially to check whether motion information that is thesame as the candidate control point motion information exists in thesubblock-based merging candidate list. If no motion information that isthe same as the candidate control point motion information exists in thesubblock-based merging candidate list, the candidate control pointmotion information is added to the subblock-based merging candidatelist.

For example, a preset order is as follows: Affine (CP1, CP2,CP3)->Affine (CP1, CP2, CP4)->Affine (CP1, CP3, CP4)->Affine (CP2, CP3,CP4)->Affine (CP1, CP2)->Affine (CP1, CP3). There are six combinationsin total.

For example, if sps_affine_type_flag=1, a preset order is as follows:Affine (CP1, CP2, CP3)->Affine (CP1, CP2, CP4)->Affine (CP1, CP3,CP4)->Affine (CP2, CP3, CP4)->Affine (CP1, CP2)->Affine (CP1, CP3).There are six combinations in total. An order for adding the sixcombinations to the candidate motion vector list is not specificallylimited in this embodiment of this application.

If sps_affine_type_flag=0, a preset order is as follows: Affine (CP1,CP2)->Affine (CP1, CP3). There are two combinations in total. An orderfor adding the two combinations to the candidate motion vector list isnot specifically limited in this embodiment of this application.

If control point motion information corresponding to a combination isunavailable, the combination is deemed unavailable. If a combination isavailable, a reference frame index of the combination is determined(when there are two control points, a minimum reference frame index isselected as the reference frame index of the combination; when there aremore than two control points, a reference frame index with a maximumpresence frequency is selected as the reference frame index of thecombination; and if a plurality of reference frame indexes have a samepresence frequency, a minimum reference frame index is selected as thereference frame index of the combination). Control point motion vectorsare further scaled. If motion information of all control points afterscaling are the same, the combination is invalid.

D3: Optionally, if sps_planar_enabled_flag=1, motion informationconstructed by using the ATMVP is added to the subblock-based mergingcandidate list. For details, refer to the foregoing descriptions in “7.Planar motion vector prediction”.

Optionally, in this embodiment of this application, the candidate motionvector list may be padded. For example, after the foregoing traversalprocess, if the length of the candidate motion vector list is less thanthe maximum list length MaxNumSubblockMergeCand, the candidate motionvector list may be padded until the list length is equal toMaxNumSubblockMergeCand.

The padding may be performed by using a zero motion vector paddingmethod, or by using a method for combining or weighted averagingexisting candidate motion information in the existing list. It should benoted that other methods for padding the candidate motion vector listmay also be used in this application. Details are not described herein.

Operation 803 b: Parse the bitstream and determine optimal control pointmotion information.

An index number merge_subblock_idx is parsed, and the optimal motioninformation is determined from the subblock-based merging candidate listbased on the index number.

merge_subblock_idx is usually binarized by using a TR code (truncatedunary, truncated unary code). In other words, merge_subblock_idx ismapped to different bin strings based on a maximum index value. Themaximum index value is preconfigured or transmitted. For example, if themaximum index value is 4, merge_subblock_idx is binarized according toTable 3.

TABLE 3 Index Bin string 0 0 1 1 0 2 1 1 0 3 1 1 1 0 4 1 1 1 1

When the maximum index value is 2, merge_subblock_idx is binarizedaccording to Table 4.

TABLE 4 Index Bin string 0 0 1 1 0 2 1 1

If merge_subblock_idx is transmitted by using a bin string, a decoderside may determine the index number based on the maximum index value andTable 2 or Table 3. For example, the maximum index value is 4. Whendecoding is performed to obtain the index number, if 0 is obtained orthe index number obtained through decoding is equal to the maximum indexvalue, the index number is determined. For example, when decoding isperformed to obtain the index number, if the first bit is 0, thedecoding that is performed to obtain the index number ends, and theindex number is determined to be 0. For another example, if the firstbit is 1 and the second bit is 0, the decoding that is performed toobtain the index number ends, and the index number is determined to be1.

Setting of the maximum index value and a binarization table is notspecifically limited in this embodiment of this application.

Operation 804 b: If the optimal motion information is ATMVP or planarmotion information, obtain a motion vector of each subblock by directlyusing the ATMVP or planar method.

If a motion mode indicated by the optimal motion information is anaffine mode, the motion vector of each subblock of the current block isobtained based on the optimal control point motion information and anaffine motion model used for a current decoding block. This process isthe same as operation 805 a.

Operation 805 b: Perform motion compensation for each subblock based onthe determined motion vector of the subblock, to obtain a predictionsample value of the subblock.

In the existing technology, there is no feasible manner of determining amaximum length (MaxNumSubblockMergeCand) of the candidate motion vectorlist corresponding to the subblock merge mode.

In view of this, the embodiments of this application provide a videopicture prediction method and apparatus, to provide a manner ofdetermining a maximum length (MaxNumSubblockMergeCand) of a candidatemotion vector list corresponding to a subblock merge mode. The methodand the apparatus are based on a same inventive concept. Because aproblem-resolving principle of the method is similar to that of theapparatus, mutual reference may be made between implementations of theapparatus and the method. No repeated descriptions are provided.

The manner of determining the maximum length of the candidate motionvector list corresponding to the subblock merge mode is described belowin detail in two cases.

In a first case, the subblock merge mode may comprise at least one of anaffine mode, an advanced temporal motion vector prediction mode, or aplanar motion vector prediction mode.

In a second case, a planar motion vector prediction mode is notconsidered to be present in the subblock merge mode. In other words, thesubblock merge mode may comprise at least one of an affine mode or anadvanced temporal motion vector prediction mode.

The following describes implementations of this application in detailfrom a perspective of a decoder side with reference to the accompanyingdrawings. Specifically, the manner may be performed by a video decoder30, or may be implemented by a motion compensation module in a videodecoder, or may be performed by a processor.

For the first case, several possible implementations, a firstimplementation to a fifth implementation, are provided as examplesaccording to some embodiments.

FIG. 9 shows descriptions of the first implementation according to someembodiments.

Operation S901: Parse a first indicator from a bitstream. Perform S902or S904.

The first indicator is used to indicate whether a candidate mode used tointer predict a to-be-processed block comprises the affine mode. Inother words, the indicator 1 is used to indicate whether the affine modecan (or is allowed to) be used to perform motion compensation on ato-be-processed block.

For example, the first indicator may be configured in an SPS in thebitstream. On this basis, parsing the first indicator from the bitstreammay be implemented in the following manner: parsing the indicator 1 fromthe SPS in the bitstream. Alternatively, the first indicator may beconfigured in a slice header of a slice in the bitstream, theto-be-processed block is comprised in the slice. Based on this, parsingthe first indicator from the bitstream may be implemented in thefollowing manner: parsing the first indicator from the slice header ofthe slice in the bitstream, the to-be-processed block is comprised inthe slice.

For example, the first indicator may be represented by a syntax elementsps_affine_enabled_flag, and sps_affine_enabled_flag is used to specifywhether an affine mode can be used to inter predict a picture blockcomprised in a video picture. For example, whensps_affine_enabled_flag=0, it indicates that the affine mode cannot beused to inter predict the picture block comprised in the video picture.When sps_affine_enabled_flag=1, it indicates that an affine motion modelcan be used to inter predict the picture block comprised in the videopicture.

Operation S902: When the first indicator indicates that the candidatemode used to inter predict the to-be-processed block comprises theaffine mode, parse a second indicator from the bitstream, where thesecond indicator is used to indicate (or determine) a maximum length ofa first candidate motion vector list, and the first candidate motionvector list is a candidate motion vector list constructed for theto-be-processed block by using a subblock merge prediction mode. Thefirst candidate motion vector list may be referred to asMaxNumSubblockMergeCand.

For example, the second indicator may be configured in the SPS, a PPS,or the slice header. Based on this, parsing the second indicator fromthe bitstream may be implemented in the following manner: parsing thesecond indicator from the sequence parameter set in the bitstream; orparsing the second indicator from the slice header of the slice,including the to-be-processed block, in the bitstream.

For example, the second indicator may be represented byK_minus_max_num_subblock_merge_cand.

For example, a value of K_minus_max_num_subblock_merge_cand is allowedto be in a range of 0 to 4.

For example, a maximum value of MaxNumSubblockMergeCand is allowed to be5.

When the maximum value of MaxNumSubblockMergeCand is allowed to be 5,the second indicator may be represented byfive_minus_max_num_subblock_merge_cand.

Table 5 shows an example of a syntax structure for parsing the secondindicator.

TABLE 5 Descriptor slice header( ) {  ...   if(sps_affine_enable_flag)   five_minus_max_num_subblock_merge_cand ue(v)  ... }

Operation S903: Determine the maximum length of the first candidatemotion vector list based on the second indicator.

In an example, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block comprises theaffine mode, the maximum length (MaxNumSubblockMergeCand) of the firstcandidate motion vector list may be obtained according to the followingformula: MaxNumSubblockMergeCand=K K_minus_max_num_subblock_merge_cand,

where MaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, and K is a preset non-negative integer.

Operation S904: When the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, and the third indicator isused to indicate that the ATMVP mode is present in the subblock mergeprediction mode, determine a third number based on the third indicator,and determine the maximum length of the first candidate motion vectorlist based on the third number.

That the candidate mode used to inter predict the to-be-processed blockonly comprises the translational motion vector prediction mode meansthat the candidate mode used to inter predict the to-be-processed blockcannot (is not allowed to) comprise the affine mode. The third indicatoris used to indicate whether the ATMVP mode is present in the subblockmerge prediction mode. In other words, the third indicator is used toindicate whether the ATMVP mode is allowed to be used to inter predictthe to-be-processed block. The third indicator may be configured in theSPS, the PPS, or the slice header.

In an example, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, and the third indicator isused to indicate that the ATMVP mode is present in the subblock mergeprediction mode, the maximum length (MaxNumSubblockMergeCand) of thefirst candidate motion vector list is equal to the third number.

For example, the third indicator may be represented bysps_sbtmvp_enabled_flag. For example, when sps_sbtmvp_enabled_flag isequal to a first value, it indicates that the ATMVP mode is not presentin the subblock merge prediction mode; or when sps_sbtmvp_enabled_flagis equal to a second value, it indicates that the ATMVP mode is presentin the subblock merge prediction mode. For example, the first value isequal to 0, and the second value is equal to 1.

For example, the third number may be used to indicate the maximum numberof motion vectors that are supported in prediction performed by usingthe ATMVP mode. For example, when the first indicator indicates that thecandidate mode used to inter predict the to-be-processed block onlycomprises the translational motion vector prediction mode, ifsps_sbtmvp_enabled_flag=0, the third number is equal to 0, or ifsps_sbtmvp_enabled_flag=1, the third number is equal to the maximumnumber of motion vectors that are supported in prediction performed byusing the ATMVP mode. In addition, when the maximum number of motionvectors that are supported in prediction performed by using the ATMVPmode is 1, the third number may be equal to a value ofsps_sbtmvp_enabled_flag. For example, if sps_sbtmvp_enabled_flag=0, thethird number is equal to 0; or if sps_sbtmvp_enabled_flag=1, the thirdnumber is equal to 1.

That the maximum value of MaxNumSubblockMergeCand is allowed to be 5 isused as an example. If sps_affine_enable_flag=0, MaxNumSubblockMergeCandis obtained according to the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag.

If sps_affine_enable_flag=1, MaxNumSubblockMergeCand is obtainedaccording to the following formula:

MaxNumSubblockMergeCand=5−five_minus_max_num_subblock_merge_cand.

For example, five_minus_max_num_subblock_merge_cand may be defined assubtracting a maximum length of a subblock-based merging motion vectorprediction list in a slice from 5(five_minus_max_num_subblock_merge_cand specifies the maximum number ofsubblock-based merging motion vector prediction (MVP) candidatessupported in the slice subtracted from 5).

The maximum number of subblock-based merging MVP candidates,MaxNumSubblockMergeCand, is derived as follows:

 If sps_affine_enabled_flag_is equal to 0: MaxNumSubblockMergeCand =sps_sbtmvp_enabled_flag;  Otherwise (sps_affine_enabled_flag is equal to1): MaxNumSubblockMergeCand = 5 −five_minus_max_num_subblock_merge_cand.

A value of MaxNumSubblockMergeCand shall be in a range of 0 to 5,inclusive.

FIG. 10 shows descriptions of the second implementation according tosome embodiments.

Operation S1001: For details of S1001, refer to S901. The details arenot described herein again. Perform S1002 or S1004.

Operation S1002: When the first indicator indicates that the candidatemode used to inter predict the to-be-processed block comprises theaffine mode, parse a second indicator and a third indicator from thebitstream, where the second indicator is used to indicate (or determine)a maximum length of a first candidate motion vector list, and the firstcandidate motion vector list is a candidate motion vector listconstructed for the to-be-processed block by using a subblock mergeprediction mode.

Operation S1003: When the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, determine a first number based on the thirdindicator, and determine the maximum length of the first candidatemotion vector list based on the second indicator and the first number.

For explanations of the first indicator, the second indicator, and thethird indicator, refer to the embodiment corresponding to FIG. 9.Details are not described herein again.

When the third indicator indicates that the ATMVP mode is not present inthe subblock merge prediction mode, the first number may be equal to themaximum number of motion vectors that are supported in predictionperformed by using the ATMVP mode. For example, the third indicator maybe represented by sps_sbtmvp_enabled_flag. Whensps_sbtmvp_enabled_flag=0, it indicates that the advanced temporalmotion vector prediction mode is not present in the subblock mergeprediction mode. In this case, the first number is equal to the maximumnumber of motion vectors that are supported in prediction performed byusing the ATMVP mode. On the contrary, when sps_sbtmvp_enabled_flag=1,it indicates that the advanced temporal motion vector prediction mode ispresent in the subblock merge prediction mode. In this case, the firstnumber is equal to 0. For example, the maximum number of motion vectorsthat are supported in prediction performed by using the ATMVP mode maybe 1. In this case, the first number may be equal to a value of thethird indicator. For details, refer to the descriptions in theembodiment corresponding to FIG. 9. The details are not described hereinagain.

In a possible example, when the first indicator indicates that thecandidate mode used to inter predict the to-be-processed block comprisesthe affine mode, the maximum length of the first candidate motion vectorlist may be obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L1, whereMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, L1 represents the first number, and Kis a preset non-negative integer.

For example, a value of K_minus_max_num_subblock_merge_cand may beallowed to be in a range of 0 to 3.

For example, a maximum value of MaxNumSubblockMergeCand is allowed to be5.

When the maximum value of MaxNumSubblockMergeCand is allowed to be 5,the second indicator may be represented byfive_minus_max_num_subblock_merge_cand.

In an example, L1 may be obtained according to the following formula:

L1=sps_sbtmvp_enabled_flag==1?0:1. If sps_sbtmvp_enabled_flag=1, L1=0.If sps_sbtmvp_enabled_flag=0, L1=1.

S1004: For details of S1004, refer to S904. The details are notdescribed herein again.

That the maximum value of MaxNumSubblockMergeCand is allowed to be 5 isused as an example. If sps_affine_enable_flag=0, MaxNumSubblockMergeCandis obtained according to the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag.

If sps_affine_enable_flag=1, MaxNumSubblockMergeCand is obtainedaccording to the following formula:

MaxNumSubblockMergeCand=5−five_minus_max_num_subblock_merge_cand−(sps_sbtmvp_enabled_flag==1?0:1).

For example, five_minus_max_num_subblock_merge_cand may be defined assubtracting a maximum length of a subblock-based merging motion vectorprediction list in a slice from 5(five_minus_max_num_subblock_merge_cand specifies the maximum number ofsubblock-based merging motion vector prediction (MVP) candidatessupported in the slice subtracted from 5). The maximum number ofsubblock-based merging MVP candidates, MaxNumSubblockMergeCand, isderived as follows:

  If sps_affine_enabled_flag is equal to 0:  MaxNumSubblockMergeCand =sps_sbtmvp_enabled_flag;   Otherwise (sps_affine_enabled_flag is equalto 1):  MaxNumSubblockMergeCand = 5 −five_minus_max_num_subblock_merge_cand (sps_sbtmvp_enabled flag == 1 ? 00 : 1).

A value of MaxNumSubblockMergeCand shall be in a range of 0 to 5,inclusive.

FIG. 11 shows descriptions of the third implementation according to someembodiments.

Operations S1101 to S1103: For details of S1101 to S1103, refer to S901to S903. The details are not described herein again.

It should be noted that, in the third implementation, a maximum value ofMaxNumSubblockMergeCand may be allowed to be 5. For example, the secondindicator may be represented by K_minus_max_num_subblock_merge_cand,where a value of K_minus_max_num_subblock_merge_cand is allowed to be ina range of 0 to 4. When the maximum value of MaxNumSubblockMergeCand isallowed to be 5, the second indicator may be represented byfive_minus_max_num_subblock_merge_cand.

Operation S1104: When the first indicator indicates that the candidatemode used to inter predict the to-be-processed block comprises only thetranslational motion vector prediction mode, parse a third indicator anda fourth indicator from the bitstream. Perform operation S1105, S1106,or S1107.

The third indicator is used to indicate a presence state of the ATMVPmode in the subblock merge prediction mode. For related descriptions ofthe third indicator, refer to the descriptions in the embodimentcorresponding to FIG. 9. Details are not described herein again.

The fourth indicator is used to indicate a presence state of thetranslational (PLANAR) motion vector prediction mode in the subblockmerge prediction mode. In other words, the fourth indicator is used toindicate whether the planar mode is allowed to be used to inter predictthe to-be-processed block.

Operation S1105: When the third indicator indicates that the ATMVP modeis present in the subblock merge prediction mode, and the fourthindicator indicates that the planar mode is not present in the subblockmerge prediction mode, determine a third number based on the thirdindicator, and determine the maximum length of the first candidatemotion vector list based on only the third number.

For example, when the fourth indicator is a third value, it indicatesthat the planar mode is not present in the subblock merge predictionmode; or when the fourth indicator is a fourth value, it indicates thatthe planar mode is present in the subblock merge prediction mode. Forexample, the third value is equal to 0, and the fourth value is equalto 1. For example, the fourth indicator may be configured in an SPS, aPPS, or a slice header. The fourth indicator may be represented bysps_planar_enabled_flag.

Operation S1106: When the third indicator indicates that the ATMVP modeis not present in the subblock merge prediction mode, and the fourthindicator indicates that the planar mode is present in the subblockmerge prediction mode, determine a fourth number based on the fourthindicator, and determine the maximum length of the first candidatemotion vector list based on the fourth number.

For example, when the fourth indicator indicates that the planar mode ispresent in the subblock merge prediction mode, the fourth number isequal to the maximum number of motion vectors that are supported inprediction performed by using the planar mode.

For example, in operation S1106, the maximum length of the firstcandidate motion vector list is equal to the fourth number. For example,if the maximum number of motion vectors that are supported in predictionperformed by using the planar mode is 1, the maximum length of the firstcandidate motion vector list is 1. For another example, when the fourthindicator is 1, it indicates that the planar mode is present in thesubblock merge prediction mode. In this case, the maximum length of thefirst candidate motion vector list is equal to the fourth indicator.

Operation S1107: When the third indicator indicates that the ATMVP modeis present in the subblock merge prediction mode, and the fourthindicator indicates that the planar mode is present in the subblockmerge prediction mode, determine a third number based on the thirdindicator, determine a fourth number based on the fourth indicator, anddetermine the maximum length of the first candidate motion vector listbased on the third number and the fourth number.

In an example, when the first indicator indicates that the candidatemode used to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, the maximum number ofmotion vectors that are supported in prediction performed by using theplanar mode is 1, and the maximum number of motion vectors that aresupported in prediction performed by using in the ATMVP mode is 1, ifthe third indicator is 1, it indicates that the ATMVP mode is present inthe subblock merge prediction mode; or if the third indicator is 0, itindicates that the ATMVP mode is not present in the subblock mergeprediction mode; and if the fourth indicator is 1, it indicates that thePLANAR mode is present in the subblock merge prediction mode; or if thefourth indicator is 0, it indicates that the planar mode is present inthe subblock merge prediction mode. In this case, the maximum length ofthe first candidate motion vector list may be equal to a sum of thethird indicator and the fourth indicator.

The third indicator is represented by sps_sbtmvp_enabled_flag, and thefourth indicator is represented by sps_planar_enabled_flag. The maximumlength of the first candidate motion vector list may be obtainedaccording to the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

When sps_sbtmvp_enabled_flag=1 (indicating that the ATMVP mode ispresent in the subblock merge prediction mode), andsps_planar_enabled_flag=0 (indicating that the planar mode is notpresent in the subblock merge prediction mode),MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag=1, which corresponds tooperation S1105. When sps_sbtmvp_enabled_flag=0 (indicating that theATMVP mode is not present in the subblock merge prediction mode), andsps_planar_enabled_flag=1 (indicating that the planar mode is present inthe subblock merge prediction mode),MaxNumSubblockMergeCand=sps_planar_enabled_flag=1, which corresponds tooperation S1106. When sps_sbtmvp_enabled_flag=1, andsps_planar_enabled_flag=1,MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag=2,which corresponds to operation S1107.

Certainly, when the first indicator indicates that the candidate modeused to inter predict the to-be-processed block only comprises thetranslational motion vector prediction mode, the third indicatorindicates that the ATMVP mode is not present in the subblock mergeprediction mode, and the fourth indicator indicates that the planar modeis not present in the subblock merge prediction mode,MaxNumSubblockMergeCand=0.

That the maximum value of MaxNumSubblockMergeCand is allowed to be 5 isused as an example.

If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtainedaccording to the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

If sps_affine_enable_flag=1, MaxNumSubblockMergeCand is obtainedaccording to the following formula:

MaxNumSubblockMergeCand=5−five_minus_max_num_subblock_merge_cand.

For example, five_minus_max_num_subblock_merge_cand may be defined assubtracting a maximum length of a subblock-based merging motion vectorprediction list in a slice from 5(five_minus_max_num_subblock_merge_cand specifies the maximum number ofsubblock-based merging motion vector prediction (MVP) candidatessupported in the slice subtracted from 5).

The maximum number of subblock-based merging MVP candidates,MaxNumSubblockMergeCand, is derived as follows:

  If sps_affine_enabled_flag is equal to 0:  MaxNumSubblockMergeCand =sps_sbtmvp_enabled_flag + sps_planar_enabled_flag;   Otherwise(sps_affine_enabled_flag is equal to 1):  MaxNumSubblockMergeCand = 5 −five_minus_max_num_subblock_merge_cand.

A value of MaxNumSubblockMergeCand shall be in a range of 0 to 5,inclusive.

FIG. 12A and FIG. 12B shows descriptions of the fourth implementationaccording to some embodiments.

Operation S1201: For details of operation S1201, refer to operationS901. The details are not described herein again. Perform operationS1202 or S1206.

Operation S1202: When the first indicator indicates that the candidatemode used to inter predict the to-be-processed block comprises theaffine mode, parse a second indicator, a third indicator, and a fourthindicator from the bitstream, where the second indicator is used toindicate (or determine) a maximum length of a first candidate motionvector list, and the first candidate motion vector list is a candidatemotion vector list constructed for the to-be-processed block by using asubblock merge prediction mode. Perform S1203, S1204, or S1205.

Operation S1203: When the third indicator indicates that the advancedtemporal motion vector prediction mode is present in the subblock mergeprediction mode, and the fourth indicator indicates that the planarmotion vector prediction mode is not present in the subblock mergeprediction mode, determine a second number based on the fourthindicator, and determine the maximum length of the first candidatemotion vector list based on the second indicator and the second number.

Operation S1204: When the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is present in the subblock mergeprediction mode, determine a first number based on the third indicator,and determine the maximum length of the first candidate motion vectorlist based on the second indicator and the first number.

Operation S1205: When the third indicator indicates that the advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode, and the fourth indicator indicates that theplanar motion vector prediction mode is not present in the subblockmerge prediction mode, determine the first number based on the thirdindicator, determine a second number based on the fourth indicator, anddetermine the maximum length of the first candidate motion vector listbased on the second indicator, the first number, and the second number.

When the fourth indicator indicates that the planar mode is not presentin the subblock merge prediction mode, the second number may be equal tothe maximum number of motion vectors that are supported in predictionperformed by using the planar mode. For example, the fourth indicatormay be represented by sps_planar_enabled_flag. Whensps_planar_enabled_flag=0, it indicates that the planar motion vectorprediction mode is not present in the subblock merge prediction mode. Inthis case, the second number is equal to the maximum number of motionvectors that are supported in prediction performed by using the ATMVPmode. On the contrary, when sps_planar_enabled_flag=1, it indicates thatthe planar motion vector prediction mode is present in the subblockmerge prediction mode. In this case, the second number is equal to 0.For example, the maximum number of motion vectors that are supported inprediction performed by using the planar mode may be 1. In this case,the second number may be equal to a value of sps_planar_enabled_flag.

In a possible example, when the first indicator indicates that thecandidate mode used to inter predict the to-be-processed block comprisesthe affine mode, the maximum length of the first candidate motion vectorlist may be obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand−L1−L2,

For example, in the fourth implementation, a value ofK_minus_max_num_subblock_merge_cand may be allowed to be in a range of 0to 2 or 0 to 3.

For example, a maximum value of MaxNumSubblockMergeCand may be allowedto be 5 or 6.

When the maximum value of MaxNumSubblockMergeCand is allowed to be 5,the second indicator may be represented byfive_minus_max_num_subblock_merge_cand. When the maximum value ofMaxNumSubblockMergeCand is allowed to be 6, the second indicator may berepresented by six_minus_max_num_subblock_merge_cand.

In an example, L1 may be obtained according to the following formula:

L1=sps_sbtmvp_enabled_flag==1?0:1. If sps_sbtmvp_enabled_flag=1, L1=0.If sps_sbtmvp_enabled_flag=0, L1=1.

In an example, L2 may be obtained according to the following formula:

L2=sps_planar_enabled_flag==1?0:1. If sps_planar_enabled_flag=1, L2=0.If sps_sbtmvp_enabled_flag=0, L2=1.

Operations S1206 to S1209: For details of operations S1206 to S1209,refer to operations S1104 to S1107. The details are not described hereinagain.

For example, the maximum value of MaxNumSubblockMergeCand may be allowedto be 5. A value of five_minus_max_num_subblock_merge_cand is in a rangeof 0 to 2.

If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtainedaccording to the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

Otherwise, (if sps_affine_enable_flag=1), MaxNumSubblockMergeCand isobtained according to the following formula:

MaxNumSubblockMergeCand=5−five_minus_max_num_subblock_merge_cand−(sps_sbtmvp_enabled_flag==1?0:1)−(sps_planar_enabled_flag==1?0:1),

where (five_minus_max_num_subblock_merge_cand specifies the maximumnumber of subblock-based merging motion vector prediction (MVP)candidates supported in the slice subtracted from 5), the maximum numberof subblock-based merging MVP candidates, MaxNumSubblockMergeCand, isderived as follows:

-   -   If sps_affine_enabled_flag is equal to 0:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag sps_planar_enabled_flag;

-   -   −Otherwise (sps_affine_enabled_flag is equal to 1):

MaxNumSubblockMergeCand=5−five_minus_max_num_subblock_merge_cand−(sps_sbtmvp_enabled_flag==1?0:1)−(sps_planar_enabled_flag==1?0:1).

A value of MaxNumSubblockMergeCand shall be in a range of 0 to 5,inclusive.

For example, the maximum value of MaxNumSubblockMergeCand may be allowedto be 6. A value of five_minus_max_num_subblock_merge_cand is in a rangeof 0 to 3.

If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtainedaccording to the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag sps_planar_enabled_flag;

Otherwise, (if sps_affine_enable_flag=1), MaxNumSubblockMergeCand isobtained according to the following formula:

MaxNumSubblockMergeCand=6−six_minus_max_num_subblock_merge_cand−(sps_sbtmvp_enabled_flag==1?0:1)−(sps_planar_enabled_flag==1?0:1).

where (six_minus_max_num_subblock_merge_cand specifies the maximumnumber of subblock-based merging motion vector prediction (MVP)candidates supported in the slice subtracted from 6), the maximum numberof subblock-based merging MVP candidates, MaxNumSubblockMergeCand, isderived as follows:

  If sps_affine_enabled_flag is equal to 0:  MaxNumSubblockMergeCand =sps_sbtmvp_enabled_flag + sps_planar enabled_flag;   Otherwise(sps_affine_enabled_flag is equal to 1):  MaxNumSubblockMergeCand = 6 −six_minus_max_num_subblock_merge_cand − (sps_sbtmvp_enabled_flag == 1 ?0 : 1) − (sps_planar_enabled_flag == 1 ? 0 : 1).

A value of MaxNumSubblockMergeCand shall be in a range of 0 to 6,inclusive.

For the second case, the planar motion vector prediction mode is notconsidered to be present in the subblock merge mode. In other words,when the subblock merge mode may comprise at least one of the affinemode and the advanced temporal motion vector prediction mode, the firstimplementation or the second implementation may be used. Details are notdescribed herein again.

Based on a same inventive concept as the method embodiments, anembodiment of this application further provides an apparatus. As shownin FIG. 13, the apparatus 1300 may be specifically a processor in avideo decoder, a chip, a chip system, or a module in a video decoder,for example, the entropy decoding unit 304 and/or the inter predictionunit 344.

For example, the apparatus may comprise a parsing unit 1301 and adetermining unit 1302. The parsing unit 1301 and the determining unit1302 perform the method operations described in the embodimentscorresponding to FIG. 9 to FIG. 12A and FIG. 12B. For example, theparsing unit 1301 may be configured to parse indicators (for example, afirst indicator, a second indicator, a third indicator, or a fourthindicator) comprised in a bitstream, and the determining unit 1302 isconfigured to determine a maximum length of a first candidate motionvector list.

An embodiment of this application further provides another structure ofan apparatus used in a decoder. As shown in FIG. 14, the apparatus 1400may comprise a communications interface 1410 and a processor 1420.Optionally, the apparatus 1400 may further comprise a memory 1430. Thememory 1430 may be disposed inside or outside the apparatus. Both theparsing unit 1301 and the determining unit 1302 shown in FIG. 13 may beimplemented by the processor 1420. The processor 1420 sends or receivesa video stream or a bitstream through the communications interface 1410,and is configured to implement the methods in FIG. 9 to FIG. 12A andFIG. 12B. In an implementation process, operations in a processingprocedure may be performed by using an integrated logic circuit ofhardware in the processor 1420 or an instruction in a form of software,to complete the methods in FIG. 9 to FIG. 12A and FIG. 12B.

The communications interface 1410 in this embodiment of this applicationmay be a circuit, a bus, a transceiver, or any other apparatus that canbe configured to exchange information. For example, the anotherapparatus may be a device connected to the apparatus 1400. For example,if the apparatus is a video encoder, the another apparatus may be avideo decoder.

In this embodiment of this application, the processor 1420 may be ageneral-purpose processor, a digital signal processor, anapplication-specific integrated circuit, a field programmable gate arrayor another programmable logic device, a discrete gate or transistorlogic device, or a discrete hardware component, and may implement orexecute the methods, operations, and logic block diagrams disclosed inthe embodiments of this application. The general-purpose processor maybe a microprocessor, any conventional processor, or the like. Theoperations of the methods disclosed with reference to the embodiments ofthis application may be directly performed by a hardware processor, ormay be performed by using a combination of hardware in the processor anda software unit. Program code executed by the processor 1420 toimplement the foregoing methods may be stored in the memory 1430. Thememory 1430 is coupled to the processor 1420.

The coupling in this embodiment of this application may be an indirectcoupling or a communication connection between apparatuses, units, ormodules in an electrical form, a mechanical form, or another form, andis used for information exchange between the apparatuses, the units, orthe modules.

The processor 1420 may operate in collaboration with the memory 1430.The memory 1430 may be a nonvolatile memory, for example, a hard diskdrive (HDD) or a solid-state drive (SSD), or may be a volatile memory,for example, a random-access memory (RAM). The memory 1430 is any othermedium that can be configured to carry or store expected program code ina form of an instruction or a data structure and that can be accessed bya computer, but is not limited thereto.

In this embodiment of this application, a specific connection mediumbetween the communications interface 1410, the processor 1420, and thememory 1430 is not limited. In this embodiment of this application, thememory 1430, the processor 1420, and the communications interface 1410are connected through a bus in FIG. 14. The bus is represented by athick line in FIG. 14. A connection mode between other components ismerely schematically described, and is not limited thereto. The bus maybe classified into an address bus, a data bus, a control bus, and thelike. For ease of representation, only one thick line is used torepresent the bus in FIG. 14, but this does not mean that there is onlyone bus or only one type of bus.

The foregoing feasible implementations and specific embodiments relatedto FIG. 9 to FIG. 12A and FIG. 12B provide descriptions of one or morevideo data decoding apparatuses in this application. It should beunderstood that, according to the foregoing descriptions, an encoderside usually determines an inter prediction mode and encodes the interprediction mode in a bitstream. After an inter prediction mode isfinally selected, an indicator (for example, the first indicator, thesecond indicator, the third indicator, or the fourth indicator describedabove) for the inter prediction mode is encoded in the bitstream byusing an encoding process that is completely inverse to the foregoingdecoding method (the encoding process corresponds to the decodingprocess of parsing the first indicator, the second indicator, the thirdindicator, or the fourth indicator). It should be understood thatdetermining a maximum length of a first candidate motion vector list bythe encoder side is completely consistent with that by the decoder side.A specific embodiment of the encoder side is not described. However, itshould be understood that the video picture prediction method describedin this application is also used in an encoding apparatus.

An embodiment of this application further provides an apparatus used inan encoder. As shown in FIG. 15, the apparatus 1500 may comprise acommunications interface 1510 and a processor 1520. Optionally, theapparatus 1500 may further comprise a memory 1530. The memory 1530 maybe disposed inside or outside the apparatus. The processor 1520 sends orreceives a video stream or a bitstream through the communicationsinterface 1510.

In one aspect, the processor 1520 is configured to: encode a firstindicator in a bitstream; and when the first indicator indicates that acandidate mode used to inter predict the to-be-processed block comprisesan affine mode, encode a second indicator in the bitstream, where thesecond indicator is used to indicate a maximum length of a firstcandidate motion vector list, and the first candidate motion vector listis a candidate motion vector list constructed for the to-be-processedblock by using a subblock merge prediction mode.

The communications interface 1510 in this embodiment of this applicationmay be a circuit, a bus, a transceiver, or any other apparatus that canbe configured to exchange information. For example, the anotherapparatus may be a device connected to the apparatus 1500. For example,if the apparatus is a video encoder, the another apparatus may be avideo decoder.

In this embodiment of this application, the processor 1520 may be ageneral-purpose processor, a digital signal processor, anapplication-specific integrated circuit, a field programmable gate arrayor another programmable logic device, a discrete gate or transistorlogic device, or a discrete hardware component, and may implement orexecute the methods, operations, and logic block diagrams disclosed inthe embodiments of this application. The general-purpose processor maybe a microprocessor, any conventional processor, or the like. Theoperations of the methods disclosed with reference to the embodiments ofthis application may be directly performed by a hardware processor, ormay be performed by using a combination of hardware in the processor anda software unit. Program code executed by the processor 1520 toimplement the foregoing methods may be stored in the memory 1530. Thememory 1530 is coupled to the processor 1520.

The coupling in this embodiment of this application may be an indirectcoupling or a communication connection between apparatuses, units, ormodules in an electrical form, a mechanical form, or another form, andis used for information exchange between the apparatuses, the units, orthe modules.

The processor 1520 may operate in collaboration with the memory 1530.The memory 1530 may be a nonvolatile memory, for example, a hard diskdrive (HDD) or a solid-state drive (SSD), or may be a volatile memory,for example, a random-access memory (RAM). The memory 1530 is any othermedium that can be configured to carry or store expected program code ina form of an instruction or a data structure and that can be accessed bya computer, but is not limited thereto.

In this embodiment of this application, a specific connection mediumbetween the communications interface 1510, the processor 1520, and thememory 1530 is not limited. In this embodiment of this application, thememory 1530, the processor 1520, and the communications interface 1510are connected through a bus in FIG. 15. The bus is represented by athick line in FIG. 15. A connection mode between other components ismerely schematically described, and is not limited thereto. The bus maybe classified into an address bus, a data bus, a control bus, and thelike. For ease of representation, only one thick line is used torepresent the bus in FIG. 15, but this does not mean that there is onlyone bus or only one type of bus.

Based on the foregoing embodiments, an embodiment of this applicationfurther provides a computer storage medium. The storage medium stores asoftware program. When the software program is read and executed by oneor more processors, the method provided in any one or more of theforegoing embodiments can be implemented. The computer storage mediummay comprise any medium that can store program code, such as a USB flashdrive, a removable hard disk, a read-only memory, a random accessmemory, a magnetic disk, or an optical disc.

Based on the foregoing embodiments, an embodiment of this applicationfurther provides a chip. The chip comprises a processor, configured toimplement the functions in any one or more of the foregoing embodiments,for example, obtaining or processing information or a message used inthe foregoing methods. Optionally, the chip further comprises a memory.The memory is configured to store a program instruction and data thatare necessary and executed by the processor. The chip may comprise achip, or may comprise a chip and another discrete device.

Although specific aspects of this application have been described withreference to the video encoder 20 and the video decoder 30, it should beunderstood that the technologies of this application may be used by manyother video encoding and/or decoding units, processors, processingunits, and hardware-based decoding units and the like, for example,encoders/decoders (CODEC). In addition, it should be understood that theoperations described and shown in FIG. 8A to FIG. 12A and FIG. 12B aremerely provided as feasible implementations. In other words, theoperations described in the feasible implementations in FIG. 8A to FIG.12A and FIG. 12B are not necessarily performed in the order shown inFIG. 8A to FIG. 12A and FIG. 12B, and fewer, additional, or alternativeoperations may be performed.

Further, it should be understood that depending on the feasibleimplementations, specific actions or events in any of the methodsdescribed in this specification may be performed in different orders, anaction or event may be added, or the actions or events may be combined,or omitted (for example, not all of the described actions or events arenecessary for implementing the methods). Further, in a particularfeasible implementation, the actions or events may (for example) undergomulti-threading processing or interrupt processing, or may be processedby a plurality of processors simultaneously instead of sequentially.Further, although specific aspects of this application are described asbeing performed by a single module or unit for the purpose of clarity,it should be understood that the technologies of this application may beperformed by a combination of units or modules associated with the videodecoder.

In one or more feasible implementations, the described functions may beimplemented by using hardware, software, firmware, or any combinationthereof. If the functions are implemented by using software, thefunctions may be stored in a computer-readable medium as one or moreinstructions or code, or transmitted through a computer-readable mediumand executed by a hardware-based processing unit. The computer-readablemedium may comprise a computer-readable storage medium or acommunications medium. The computer-readable storage medium correspondsto a tangible medium such as a data storage medium. The communicationsmedium comprises any medium that facilitates transmission of a computerprogram (for example) from one location to another location according toa communications protocol.

In this manner, the computer-readable medium may correspond to, forexample, (1) a non-transitory tangible computer-readable storage medium,or (2) a communications medium such as a signal or a carrier. The datastorage medium may be any available medium that can be accessed by oneor more computers or one or more processors to retrieve instructions,code, and/or data structures for implementing the technologies describedin this application. A computer program product may comprise acomputer-readable medium.

By way of a feasible implementation rather than a limitation, thecomputer-readable storage medium may comprise a RAM, a ROM, an EEPROM, aCD-ROM or another optical disk storage apparatus, a magnetic diskstorage apparatus or another magnetic storage apparatus, a flash memory,or any other medium that can be used to store required code in a form ofan instruction or a data structure and that can be accessed by acomputer. Likewise, any connection may be appropriately referred to as acomputer-readable medium. For example, if an instruction is transmittedfrom a website, server, or another remote source through a coaxialcable, an optical fiber, a twisted pair, a digital subscriber line(DSL), or wireless technologies such as infrared, radio, and microwave,the coaxial cable, optical fiber, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are comprised in adefinition of medium.

However, it should be understood that the computer-readable storagemedium and the data storage medium may not comprise a connection, acarrier, a signal, or another transitory medium, but alternatively meannon-transitory tangible storage media. A magnetic disk and an opticaldisc described in this specification comprise a compact disc (CD), alaser disc, an optical disc, a digital versatile disc (DVD), a floppydisk, and a Blu-ray disc. The magnetic disk usually reproduces datamagnetically, and the optical disc reproduces data optically through alaser. A combination of the foregoing magnetic disk and optical discshall also be comprised in a scope of the computer-readable medium.

An instruction may be executed by one or more processors such as one ormore digital signal processors (DSP), general-purpose microprocessors,application-specific integrated circuits (ASIC), field programmable gatearrays (FPGA), or other equivalent integrated or discrete logiccircuits. Therefore, the term “processor” used in this specification maybe any one of the foregoing structures or another structure that is usedto implement the technologies described in this specification. Inaddition, in some aspects, the functions described in this specificationmay be provided within dedicated hardware and/or software modulesconfigured for encoding and decoding, or may be incorporated into acombined codec. Likewise, the technologies may all be implemented in oneor more circuits or logic elements.

The technologies in this application may be implemented in variousapparatuses or devices, including a wireless mobile phone, an integratedcircuit (IC), or a set of ICs (for example, a chip set). Variouscomponents, modules, or units are described in this application toemphasize functional aspects of an apparatus configured to perform thedisclosed technologies, but are not necessarily implemented by differenthardware units. More precisely, as described above, various units may becombined into a codec hardware unit or provided by interoperablehardware units (including one or more processors described above) incombination with an appropriate software and/or firmware set.

The foregoing descriptions are merely examples of specificimplementations of this application, but are not intended to limit theprotection scope of this application. Any variation or replacementreadily figured out by a person skilled in the art within the technicalscope disclosed in this application shall fall within the protectionscope of this application. Therefore, the protection scope of thisapplication shall be subject to the protection scope of the claims.

What is claimed is:
 1. A video picture prediction method, comprising:parsing, from a sequence parameter set (SPS) in a bitstream, a value ofa first indicator; if the value of the first indicator is equal to afirst value, parsing a value of a second indicator from the SPS, anddetermining a maximum length of a first candidate motion vector listbased on the value of the second indicator, wherein the first candidatemotion vector list is constructed for a block to be processed, wherein asubblock merge prediction mode is used for the block; and if the valueof the first indicator is equal to a second value, determining themaximum length of the first candidate motion vector list is 0, whereinthe second value is different from the first value.
 2. The method ofclaim 1, wherein the maximum length of the first candidate motion vectorlist is obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand, whereinMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, and K is a preset non-negative integer.3. The method of claim 1, wherein an advanced temporal motion vectorprediction mode is not present in the subblock merge prediction mode. 4.The method of claim 1, wherein the value of the first indicatorindicates whether a candidate mode used to inter predict the blockcomprises an affine mode.
 5. The method of claim 1, wherein the firstvalue is 1, the second value is
 0. 6. A video decoder, comprising: oneor more processors; and a non-transitory computer-readable storagemedium coupled to the processors and storing programming for executionby the processors, wherein the programming, when executed by theprocessors, configures the decoder to carry out operations of parsing,from a sequence parameter set (SPS) in a bitstream, a value of a firstindicator; if the value of the first indicator is equal to a firstvalue, parsing a value of a second indicator from the SPS, anddetermining a maximum length of a first candidate motion vector listbased on the value of the second indicator, wherein the first candidatemotion vector list is constructed for a block to be processed, asubblock merge prediction mode is used for the block; and if the valueof the first indicator is equal to a second value, determining themaximum length of the first candidate motion vector list is 0, thesecond value is different from the first value.
 7. The decoder of claim6, wherein the maximum length of the first candidate motion vector listis obtained according to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand, whereinMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, and K is a preset non-negative integer.8. The decoder of claim 6, wherein an advanced temporal motion vectorprediction mode is not present in the subblock merge prediction mode. 9.The decoder of claim 6, wherein the value of the first indicatorindicates whether a candidate mode used to inter predict the blockcomprises an affine mode.
 10. The decoder of claim 6, wherein the firstvalue is 1, the second value is
 0. 11. A video data decoding device,comprising: a non-transitory memory storage, configured to store videodata in a form of a bitstream; and the video data decoding device isconfigured to perform the operations of parsing, from a sequenceparameter set (SPS) in a bitstream, a value of a first indicator; if thevalue of the first indicator is equal to a first value, parsing a valueof a second indicator from the SPS, and determining a maximum length ofa first candidate motion vector list based on the value of the secondindicator, wherein the first candidate motion vector list is constructedfor a block to be processed, wherein a subblock merge prediction mode isused for the block; and if the value of the first indicator is equal toa second value, determining the maximum length of the first candidatemotion vector list is 0, wherein the second value is different from thefirst value.
 12. The video data decoding device of claim 11, wherein themaximum length of the first candidate motion vector list is obtainedaccording to the following formula:MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand, whereinMaxNumSubblockMergeCand represents the maximum length of the firstcandidate motion vector list, K_minus_max_num_subblock_merge_candrepresents the second indicator, and K is a preset non-negative integer.13. The video data decoding device of claim 11, wherein an advancedtemporal motion vector prediction mode is not present in the subblockmerge prediction mode.
 14. The video data decoding device of claim 11,wherein the value of the first indicator indicates whether a candidatemode used to inter predict the block comprises an affine mode.
 15. Thevideo data decoding device of claim 11, wherein the first value is 1,the second value is 0.